How to Resolve Intermittent 403 Errors in GitLab on Ubuntu 14.04

System Environment

System information
System: Ubuntu 14.04
Current User: git
Using RVM: no
Ruby Version: 2.1.5p273
Gem Version: 2.2.1
Bundler Version: 1.5.3
Rake Version: 10.3.2
Sidekiq Version: 3.3.0
GitLab information
Version: 7.8.1
Revision: e2d785c
Directory: /opt/gitlab/embedded/service/gitlab-rails
DB Adapter: postgresql
URL: https://git.zhuima.com
HTTP Clone URL: https://git.zhuima.com/some-project.git
SSH Clone URL: ssh://[email protected]:10086/zhuima.git
Using LDAP: yes
Using Omniauth: no

GitLab Shell
Version: 2.5.4
Repositories: /data/gitlab/data/repositories
Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks/
Git: /opt/gitlab/embedded/bin/git

Problem Symptoms

Every day there is a period when a 403 error occurs and the office network cannot access GitLab.

Debugging Idea

1. Check logs for 401 or 403 status codes.
2. Run `gitlab-rake gitlab:check` and apply suggested fixes, e.g.:
   sudo -u git -H bundle exec rake gitlab:satellites:create RAILS_ENV=production
   (remove tmp/repo_satellites if necessary and rerun)
3. Search online documentation for related issues.

Google Search Findings

http://boardreader.com/thread/Gitlab_7_10_4_Forbidden_Error_56o55lX769.html
Suspect that the policy in `rack_attack.rb` is causing the problem; modifications and restart did not take effect.

Changing the `rack_attack.rb` configuration to allow 1‑second bursts of 300 requests did not work.

unless Rails.env.test?
  Rack::Attack.throttle('protected paths', limit: 300, period: 1.seconds) do |req|
    if req.post? && req.path =~ paths_regex
      req.ip
    end
  end
end

Other Observations

Each service interruption lasts about 1 hour.
No scheduled tasks are set during this period.
GitLab authenticates accounts via LDAP.
Only the office network cannot access GitLab; external network works fine.

Related Description

1. Only the office network cannot access GitLab, external network works.
2. Office network has >150 developers.
3. Concurrency >30.

Desired Outcome

1. How to disable the `rack_attack.rb` policy.
2. Achieve stable service.

Related Logs

Started GET "/" for 127.0.0.1 at 2015-09-02 16:57:50 +0800
Processing by DashboardController#show as */*
Completed 401 Unauthorized in 44ms

/var/log/gitlab/nginx/gitlab_access.log
118.187.12.36 - - [02/Sep/2015:16:47:33 +0800] "GET /zhuima-egg/zhuima.git/info/refs?service=git-upload-pack HTTP/1.1" 403 20 "-" "git/1.9.5.msysgit.1"

Operations Performed During Debugging

Added whitelist entries in `1_settings.rb`; changes took effect but were overwritten after `gitlab-ctl reconfigure`.

Settings['rack_attack'] ||= Settingslogic.new({})
Settings.rack_attack['git_basic_auth'] ||= Settingslogic.new({})
Settings.rack_attack.git_basic_auth['enabled'] = true if Settings.rack_attack.git_basic_auth['enabled'].nil?
Settings.rack_attack.git_basic_auth['ip_whitelist'] ||= %w{127.0.0.1}
Settings.rack_attack.git_basic_auth['ip_whitelist'] ||= %w{118.187.12.36}
Settings.rack_attack.git_basic_auth['maxretry'] ||= 10
Settings.rack_attack.git_basic_auth['findtime'] ||= 1.minute
Settings.rack_attack.git_basic_auth['bantime'] ||= 1.hour

Attempted to disable `rack_attack.rb` by removing the file.

http://boardreader.com/thread/Gitlab_7_10_4_Forbidden_Error_56o55lX769.html

About rack_attack

rack-attack: a Rack middleware for blocking abusive requests.
https://github.com/kickstarter/rack-attack
https://github.com/kickstarter/rack-attack/wiki/Example-Configuration

Official Documentation Explanation

https://doc.gitlab.com/ce/security/rack_attack.html
https://boardreader.com/thread/Gitlab_7_10_4_Forbidden_Error_56o55lX769.html

rack_attack Configuration Example (GitLab 7.9+)

https://github.com/kickstarter/rack-attack/wiki/Example-Configuration

Reference Documents

https://code.csdn.net/zhanglushan/gitlabhq/tree/0b1cf50060de0d0a3039dfb2fca47364c7fb5f