Skip to content

[Solved] GitLab Cannot Startup and Freeze at "systemctl start gitlab-runsvdir.service"

Last Updated on 2022-07-15 by Clay

Today I encountered a problem about the GitLab cannot startup. It freeze at the sudo systemctl start gitlab-runsvdir.service command.

Either use:

sudo gitlab-ctl status
sudo gitlab-ctl stop
sudo gitlab-ctl start


Will appear:

fail: alertmanager: runsv not running
fail: crond: runsv not running
fail: gitaly: runsv not running
fail: gitlab-exporter: runsv not running
fail: gitlab-workhorse: runsv not running
fail: grafana: runsv not running
fail: logrotate: runsv not running
fail: nginx: runsv not running
fail: node-exporter: runsv not running
fail: postgres-exporter: runsv not running
fail: postgresql: runsv not running
fail: prometheus: runsv not running
fail: redis: runsv not running
fail: redis-exporter: runsv not running
fail: registry: runsv not running
fail: sidekiq: runsv not running
fail: unicorn: runsv not running


So even if I want to restart GitLab, I can't.

Solution

Perhaps there is a possibility that it is blocked by the plymouth-quit-wait.service, just like: https://forum.gitlab.com/t/plymouth-quit-wait-service-preventing-gitlab-runsvdir-service-possible/48442.

The poster mentioned that he spent two hours starting his GitLab CE, and in the end he locked plymouth-quit-wait.service as the problem.

And I am in the same situation. After I use the following command, I can see many waiting services. 

sudo systemctl list-jobs


Output:

JOB UNIT                                 TYPE  STATE  
 98 gitlab-runsvdir.service              start waiting
  1 graphical.target                     start waiting
120 systemd-update-utmp-runlevel.service start waiting
182 ureadahead-stop.timer                start waiting
104 plymouth-quit-wait.service           start running
174 system-getty.slice                   start waiting
 14 setvtrgb.service                     start waiting
  2 multi-user.target                    start waiting

8 jobs listed.

They are waiting for plymouth-quit-wait.service number 104.

Then I checked the instructions on the Internet and learned that this service is only to load the startup icon during the boot process, and it should be finished at the end of the boot.

However, before executing the following commands, please consider whether there are any information security risks. Hackers could exploit vulnerabilities that GitLab has not yet updated.

sudo systemctl stop plymouth-quit-wait.service

sudo systemctl start gitlab-runsvdir.service
sudo systemctl list-jobs


Output:

No jobs running.


Verify that no trips are blocking, then try to start gitlab:

sudo gitlab-ctl start


Output:

ok: run: alertmanager: (pid 28589) 31s
ok: run: crond: (pid 28570) 31s
ok: run: gitaly: (pid 28574) 31s
ok: run: gitlab-exporter: (pid 28571) 31s
ok: run: gitlab-workhorse: (pid 28586) 31s
ok: run: grafana: (pid 28588) 31s
ok: run: logrotate: (pid 28566) 31s
ok: run: nginx: (pid 28562) 31s
ok: run: node-exporter: (pid 28572) 31s
ok: run: postgres-exporter: (pid 28585) 31s
ok: run: postgresql: (pid 28565) 31s
ok: run: prometheus: (pid 28575) 31s
ok: run: redis: (pid 28569) 31s
ok: run: redis-exporter: (pid 28577) 31s
ok: run: registry: (pid 28578) 31s
ok: run: sidekiq: (pid 28584) 31s
ok: run: unicorn: (pid 28587) 31s

This time there was no problem. Perhaps this is a solution that can be referred to, and I am very grateful to the sender who left the record.

References

Read More

Tags:

4 thoughts on “[Solved] GitLab Cannot Startup and Freeze at "systemctl start gitlab-runsvdir.service"”

Leave a Reply