Конфигурирование Puma и Sidekiq

Возможно, вам придется больше пытаться преодолеть кривую обучения + актуальный код вопроса. Я прошу прощения, если это кажется скучным, в настоящее время я получаю эту ошибку в процессе производства.Конфигурирование Puma и Sidekiq

В основном я продолжаю считать, что этот «пул redis слишком мал», и я теряюсь, с чего начать, я действительно потерял понимание в основном того, как точно настроить sidekiq с помощью puma или что-нибудь, что приходит после конфигурации, например масштабирования и т. Д.

Ниже я получил свою конфигурацию, следуя за моей ошибкой, которую я получаю.

PROCFILE веб: расслоение EXEC пума -C конфигурации/puma.rb работника: расслоение EXEC sidekiq -e производства -C конфигурации/sidekiq.yml

Sidekiq иницилизации

if Rails.env.production? 

    Sidekiq.configure_client do |config| 
    config.redis = { url: ENV['REDIS_URL'], size: 2 } 
    end 

    Sidekiq.configure_server do |config| 
    config.redis = { url: ENV['REDIS_URL'], size: 20 } 

    Rails.application.config.after_initialize do 
     Rails.logger.info("DB Connection Pool size for Sidekiq Server before disconnect is: #{ActiveRecord::Base.connection.pool.instance_variable_get('@size')}") 
     ActiveRecord::Base.connection_pool.disconnect! 

     ActiveSupport.on_load(:active_record) do 
     config = Rails.application.config.database_configuration[Rails.env] 
     config['reaping_frequency'] = ENV['DATABASE_REAP_FREQ'] || 10 # seconds 
     # config['pool'] = ENV['WORKER_DB_POOL_SIZE'] || Sidekiq.options[:concurrency] 
     config['pool'] = 16 
     ActiveRecord::Base.establish_connection(config) 

     Rails.logger.info("DB Connection Pool size for Sidekiq Server is now: #{ActiveRecord::Base.connection.pool.instance_variable_get('@size')}") 
     end 
    end 
    end 

end

Redis инициализации

if ENV["REDISTOGO_URL"] 
    $redis = Redis.new(:url => ENV["REDISTOGO_URL"]) 
    end

puma.rb

workers Integer(ENV['WEB_CONCURRENCY'] || 2) 
    threads_count = Integer(ENV['MAX_THREADS'] || 1) 
    threads threads_count, threads_count 

    preload_app! 

    rackup  DefaultRackup 
    port  ENV['PORT']  || 3000 
    environment ENV['RACK_ENV'] || 'development' 

    # Because we are using preload_app, an instance of our app is created by master process (calling our initializers) and then memory space 
    # is forked. So we should close DB connection in the master process to avoid connection leaks. 
    # https://github.com/puma/puma/issues/303 
    # http://stackoverflow.com/questions/17903689/puma-cluster-configuration-on-heroku 
    # http://www.rubydoc.info/gems/puma/2.14.0/Puma%2FDSL%3Abefore_fork 
    # Dont have to worry about Sidekiq's connection to Redis because connections are only created when needed. As long as we are not 
    # queuing workers when rails is booting, there will be no redis connections to disconnect, so it should be fine. 
    before_fork do 
     puts "Puma master process about to fork. Closing existing Active record connections." 
     ActiveRecord::Base.connection.disconnect! 
    end 

    on_worker_boot do 
     # Worker specific setup for Rails 4.1+ 
     # See: https://devcenter.heroku.com/articles/deploying-rails-applications-with-the-puma-web-server#on-worker-boot 
     ActiveRecord::Base.establish_connection 
    end

ошибка в логах

   Your Redis connection pool is too small for Sidekiq to work. Your pool has 20 connections but really needs to have at least 22 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/sidekiq-4.1.4/lib/sidekiq/redis_connection.rb:38:in `verify_sizing' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/sidekiq-4.1.4/lib/sidekiq/redis_connection.rb:17:in `create' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/sidekiq-4.1.4/lib/sidekiq.rb:128:in `redis=' 
10:04:44 worker.1 | /home/ubuntu/workspace/sample_v1/config/initializers/sidekiq.rb:10:in `block in <top (required)>' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/sidekiq-4.1.4/lib/sidekiq.rb:70:in `configure_server' 
10:04:44 worker.1 | /home/ubuntu/workspace/sample_v1/config/initializers/sidekiq.rb:9:in `<top (required)>' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/activesupport-4.2.6/lib/active_support/dependencies.rb:268:in `load' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/activesupport-4.2.6/lib/active_support/dependencies.rb:268:in `block in load' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/activesupport-4.2.6/lib/active_support/dependencies.rb:240:in `load_dependency' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/activesupport-4.2.6/lib/active_support/dependencies.rb:268:in `load' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.6/lib/rails/engine.rb:652:in `block in load_config_initializer' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/activesupport-4.2.6/lib/active_support/notifications.rb:166:in `instrument' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.6/lib/rails/engine.rb:651:in `load_config_initializer' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.6/lib/rails/engine.rb:616:in `block (2 levels) in <class:Engine>' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.6/lib/rails/engine.rb:615:in `each' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.6/lib/rails/engine.rb:615:in `block in <class:Engine>' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.6/lib/rails/initializable.rb:30:in `instance_exec' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.6/lib/rails/initializable.rb:30:in `run' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.6/lib/rails/initializable.rb:55:in `block in run_initializers' 
10:04:44 worker.1 | /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:228:in `block in tsort_each' 
10:04:44 worker.1 | /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:350:in `block (2 levels) in each_strongly_connected_component' 
10:04:44 worker.1 | /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:422:in `block (2 levels) in each_strongly_connected_component_from' 
10:04:44 worker.1 | /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:431:in `each_strongly_connected_component_from' 
10:04:44 worker.1 | /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:421:in `block in each_strongly_connected_component_from' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.6/lib/rails/initializable.rb:44:in `each' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.6/lib/rails/initializable.rb:44:in `tsort_each_child' 
10:04:44 worker.1 | /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:415:in `call' 
10:04:44 worker.1 | /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:415:in `each_strongly_connected_component_from' 
10:04:44 worker.1 | /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:349:in `block in each_strongly_connected_component' 
10:04:44 worker.1 | /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:347:in `each' 
10:04:44 worker.1 | /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:347:in `call' 
10:04:44 worker.1 | /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:347:in `each_strongly_connected_component' 
10:04:44 worker.1 | /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:226:in `tsort_each' 
10:04:44 worker.1 | /usr/local/rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/tsort.rb:205:in `tsort_each' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.6/lib/rails/initializable.rb:54:in `run_initializers' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/railties-4.2.6/lib/rails/application.rb:352:in `initialize!' 
10:04:44 worker.1 | /home/ubuntu/workspace/sample_v1/config/environment.rb:5:in `<top (required)>' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/activesupport-4.2.6/lib/active_support/dependencies.rb:274:in `require' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/activesupport-4.2.6/lib/active_support/dependencies.rb:274:in `block in require' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/activesupport-4.2.6/lib/active_support/dependencies.rb:240:in `load_dependency' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/activesupport-4.2.6/lib/active_support/dependencies.rb:274:in `require' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/sidekiq-4.1.4/lib/sidekiq/cli.rb:237:in `boot_system' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/sidekiq-4.1.4/lib/sidekiq/cli.rb:50:in `run' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/sidekiq-4.1.4/bin/sidekiq:12:in `<top (required)>' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/bin/sidekiq:23:in `load' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/bin/sidekiq:23:in `<top (required)>' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/bundler-1.12.3/lib/bundler/cli/exec.rb:63:in `load' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/bundler-1.12.3/lib/bundler/cli/exec.rb:63:in `kernel_load' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/bundler-1.12.3/lib/bundler/cli/exec.rb:24:in `run' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/bundler-1.12.3/lib/bundler/cli.rb:304:in `exec' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/bundler-1.12.3/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/bundler-1.12.3/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/bundler-1.12.3/lib/bundler/vendor/thor/lib/thor.rb:359:in `dispatch' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/bundler-1.12.3/lib/bundler/vendor/thor/lib/thor/base.rb:440:in `start' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/bundler-1.12.3/lib/bundler/cli.rb:11:in `start' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/bundler-1.12.3/exe/bundle:27:in `block in <top (required)>' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/bundler-1.12.3/lib/bundler/friendly_errors.rb:98:in `with_friendly_errors' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/gems/bundler-1.12.3/exe/bundle:19:in `<top (required)>' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/bin/bundle:23:in `load' 
10:04:44 worker.1 | /usr/local/rvm/gems/ruby-2.3.0/bin/bundle:23:in `<main>' 
10:04:45 worker.1 | exited with code 1 
10:04:45 system | sending SIGTERM to all processes

Любая помощь или своего рода руководством будет удивительно.

источник

2016-07-10 leafshinobi25

Удалите атрибут size: в блоках configure_ *. Пусть Sidekiq управляет размером для вас.

источник

2016-07-10 18:26:32

Спасибо! @Mike Perham – leafshinobi25

Я получаю сообщение об ошибке, так как я отключил атрибут size, я обновил свой код внутри gist вместе с ошибками, которые я получаю. Когда я возвращаю атрибут размера, похоже, что redis работает нормально, но действия, которые обрабатываются в фоновом режиме, например, почтовые программы не перемещаются из очереди. Я настроил активную работу и почтовую программу с sidekiq в соответствии с примерами и играл с попыткой выполнить очереди, включая создание другого типа, чтобы он не работал со значением по умолчанию. – leafshinobi25

Это ссылка на Gist, любая помощь была бы действительно оценена. https://gist.github.com/anonymous/1a63dc3568bda86097b0822bd66b05f8 – leafshinobi25

Конфигурирование Puma и Sidekiq

ответ

Смежные вопросы