Disabling Discourse IP Rate Limiting

Neo · December 4, 2020, 5:59am

Not fully sure why this site is not indexed by Googlebot as completely as the legacy site; but I did notice that the Discourse rate limiters per IP are much lower than the rate which Googlebot used to index our site.

So, in an experiment to see if the site gets indexed more, today I set this setting to:

DISCOURSE_MAX_REQS_PER_IP_MODE: 'warn'

discourse/discourse/blob/0a0fd6eacee905fe16bc3cee7884a0e1c15cbf3b/lib/middleware/request_tracker.rb

# frozen_string_literal: true

require 'method_profiler'
require 'middleware/anonymous_cache'

class Middleware::RequestTracker

  @@detailed_request_loggers = nil
  @@ip_skipper = nil

  # register callbacks for detailed request loggers called on every request
  # example:
  #
  # Middleware::RequestTracker.detailed_request_logger(->|env, data| do
  #   # do stuff with env and data
  # end
  def self.register_detailed_request_logger(callback)
    MethodProfiler.ensure_discourse_instrumentation!
    (@@detailed_request_loggers ||= []) << callback
  end

This file has been truncated. show original

Let's see what happens.

Maybe we need to turn it back on later? Let's experiment!