Topic: PHP-FPM 5.4.5 CentOS 5

PHP-FPM upgrade installs correctly, and all of my site works, however, when using the php-fpm init.d startup script (or even just the php-fpm command to control the daemon) the command line just hangs.

For example on "service php-fpm restart" it will say "Stopping php-fpm: [FAILED]
Starting php-fpm: " and it will stay blank.

I have checked my permissions, pid file, etc, error logs and its all working (no error logs at all). php-fpm does seem to restart, but the console just hangs and I have to disconnect my remote ssh console to do any more commands.

It seems to me a problem with the daemon/original command itself, as opposed to the startup script. Even just running "php-fpm" causes the console to hang forever. I have tried disabling the daemonize option in php-fpm but this makes no difference, same error.

Re: PHP-FPM 5.4.5 CentOS 5

more information ? distro ? version ? arch ? config ?

Laptop:  Fedora 38 + rpmfusion + remi (SCL only)
x86_64 builder: Fedora 39 + rpmfusion + remi-test
aarch64 builder: RHEL 9 with EPEL
Hosting Server: CentOS 8 Stream with EPEL, rpmfusion, remi

3 (edited by austinh 2012-08-03 08:54:20)

Re: PHP-FPM 5.4.5 CentOS 5

CentOS 5.6, x86_64 PHP 5.4.5 (remi-test)

1GB RAM, installing from YUM.

Previous version worked fine. All other php-extensions work fine. php-fpm even works correctly like i said before, but it just wont accurately run the init.d daemon start script without hanging.

Also I know this isn't necessarily to do with your builds Remi but someone else is having the same error at http://www.dotdeb.org/2012/07/23/php-5- … hp-5-3-15/

Half way down the comments Daniel Caillibaud describes a similar issue with the same version. Perhaps it is a php bug?

Re: PHP-FPM 5.4.5 CentOS 5

Hum...

# rpm -q php-fpm
php-fpm-5.4.5-1.el5.remi

# php-fpm -v
PHP 5.4.5 (fpm-fcgi) (built: Jul 20 2012 09:55:50)
Copyright (c) 1997-2009 The PHP Group
Zend Engine v2.4.0, Copyright (c) 1998-2012 Zend Technologies

# service php-fpm start
Démarrage de php-fpm :                                     [  OK  ]

# service php-fpm status
php-fpm (pid  8101) en cours d'exécution...

# service php-fpm stop
Arrêt de php-fpm :                                         [  OK  ]
Laptop:  Fedora 38 + rpmfusion + remi (SCL only)
x86_64 builder: Fedora 39 + rpmfusion + remi-test
aarch64 builder: RHEL 9 with EPEL
Hosting Server: CentOS 8 Stream with EPEL, rpmfusion, remi

Re: PHP-FPM 5.4.5 CentOS 5

Yes, I have seen a similar problem in PHP v5.3.15 aswell.
It seems to be a bug in php-fpm, I compared the INIT scripts to version 5.3.13 and they where identical, but that version worked fine...

http://groups.google.com/group/xcache/b … 43226d5686

Re: PHP-FPM 5.4.5 CentOS 5

[root~]# php-fpm -v
PHP 5.4.5 (fpm-fcgi) (built: Jul 20 2012 09:55:50)
Copyright (c) 1997-2009 The PHP Group
Zend Engine v2.4.0, Copyright (c) 1998-2012 Zend Technologies
[root~]# service php-fpm start
Starting php-fpm:
[root~]# service php-fpm status
php-fpm (pid  15676) is running...
[root~]# service php-fpm stop
Stopping php-fpm:                                          [FAILED]
[root~]# service php-fpm status
php-fpm dead but subsys locked
[root~]# service php-fpm start
Starting php-fpm:

It just hangs. When it says FAILED however, it still seems to accurately stop the as shown with the "dead" message. Starting it up again does start it up, but the current console hangs forever. Even "Ctrl+C" or "D" doesn't do anything.

Re: PHP-FPM 5.4.5 CentOS 5

> CentOS 5.6, x86_64
Do you encounter the same issue with a full updated distro (5.9 + all updates) ?

Laptop:  Fedora 38 + rpmfusion + remi (SCL only)
x86_64 builder: Fedora 39 + rpmfusion + remi-test
aarch64 builder: RHEL 9 with EPEL
Hosting Server: CentOS 8 Stream with EPEL, rpmfusion, remi

8 (edited by lewellyn 2012-08-17 22:06:49)

Re: PHP-FPM 5.4.5 CentOS 5

For what it's worth, I also see this upon occasion on Scientific Linux 6.3 with 5.3.

Luckily, it doesn't seem to happen during the boot process (though my servers have LOM access, it's a pain to SSH into the actual console or use a Java applet while remotely rebooting). I suspect there's a race going on, as it seems that not all of the prior php-fpm processes have quit yet when this occurs.

My workaround is to (as root):

# pkill php-fpm && pkill -9 php-fpm

This attempts to gracefully quit anything that can be (such as the service command that's trying to spawn processes), and forcefully quit anything that's hung. It "works", but it also seems that a bit of patience also does the trick.

This also seems to happen more on servers with busier (in terms of PHP usage) sites, and with multiple php-fpm pools. I haven't really investigated it fully, but it's not unique to EL5 or PHP 5.4. I suspect it's also not unique to Remi's build, but that people on e.g. Fedora don't really do "servery" stuff and it doesn't get noticed as much. In the "servery" world, php-fpm is still a scary new thing. wink

For what it's worth, this is what shows up on the "service php-fpm restart" terminal after the first kill command, quoted above:

Starting php-fpm: /sbin/service: line 66: 19476 Terminated              env -i PATH="$PATH" TERM="$TERM" "${SERVICEDIR}/${SERVICE}" ${OPTIONS}

And the obligatory additional information:

$ lsb_release -d && uname -srvmpio && php-fpm -v && repoquery --installed -i php-fpm
Description:    Scientific Linux release 6.3 (Carbon)
Linux 2.6.32-279.2.1.el6.x86_64 #1 SMP Thu Jul 19 16:08:55 CDT 2012 x86_64 x86_64 x86_64 GNU/Linux
PHP 5.3.16 (fpm-fcgi) (built: Aug 16 2012 13:52:44)
Copyright (c) 1997-2009 The PHP Group
Zend Engine v2.3.0, Copyright (c) 1998-2012 Zend Technologies
    with Xdebug v2.1.4, Copyright (c) 2002-2012, by Derick Rethans

Name        : php-fpm
Version     : 5.3.16
Release     : 1.el6.remi
Architecture: x86_64
Size        : 3392329
Packager    : http://blog.famillecollet.com/
Group       : Development/Languages
URL         : I had to delete php's link, since the forum complained about it. Sigh.
Repository  : installed
Summary     : PHP FastCGI Process Manager
Source      : php-5.3.16-1.el6.remi.src.rpm
Description :
PHP-FPM (FastCGI Process Manager) is an alternative PHP FastCGI
implementation with some additional features useful for sites of
any size, especially busier sites.

EDIT: I just updated a host that experiences the issue quite often (the same one I gave the uname and such for) to 5.4.6 from remi-test, and I can't make it happen again. That's the joy of intermittent bugs, eh? I'll update again if I can get it to reproduce with 5.4.

Re: PHP-FPM 5.4.5 CentOS 5

Thanks for this comment

I suspect it's also not unique to Remi's build

Probably worth an upstream bug report ?

Fedora don't really do "servery" stuff and it doesn't get noticed as much.

Perhaps... I get a lot of feedback from "EL + remi repo", probably more than from "Fedora".
Sometime issue comes from an system library, which are often more recent on fedora than on EL.

I just updated a host that experiences the issue quite often (the same one I gave the uname and such for) to 5.4.6 from remi-test, and I can't make it happen again.

Great !

Do you use other extensions (I think of APC) ?
I know latest APC version 3.1.12 fixes some bug with FPM (and introduce some new ones, with stat...)

Laptop:  Fedora 38 + rpmfusion + remi (SCL only)
x86_64 builder: Fedora 39 + rpmfusion + remi-test
aarch64 builder: RHEL 9 with EPEL
Hosting Server: CentOS 8 Stream with EPEL, rpmfusion, remi

Re: PHP-FPM 5.4.5 CentOS 5

It's actually possible to reproduce the bug on PHP 5.4 aswell but it is much more frequently occurring on PHP 5.3
I have no idea why, opcode cachers do not cause the problem per se, but they make it occur more frequently, probably due to the increased concurrency...

Re: PHP-FPM 5.4.5 CentOS 5

I just had it happen on 5.4.6 finally. I don't, as far as I know offhand, use any opcode caching.

What it appears is happening is that for some reason, one of the php-fpm processes becomes a zombie. It seems that if I manually kill the zombie's parent, things shut down properly. I'll keep an eye out for it to happen again, and hopefully I can use that opportunity to probe at it some and maybe even figure out what might help at least prevent the visible effects of this issue (since it's possible that the cause is in the PHP core, and may turn out to be benign).

I haven't had a chance to check on this thread in the past couple days, but this was the first time it's happened since upgrading to 5.4. That meshes with Per Hansson's note that it's more common on 5.3.

Re: PHP-FPM 5.4.5 CentOS 5

I've got the same problem after updating php-5.4.3 to php-5.4.6 on my CentOS 6.3 vserver. Both php versions are self compiled. With php-5.4.3 and previous versions I never had this problem.

I recognized that the terminal in the ssh session was frozen after starting the php-fpm service, killed the connection and within my new connection I could verify that the php-fpm processes are running.

BUT now the _critical_ point. While rebooting the vserver I was not able to connect per ssh. The reason was, that the frozen php-fpm stopped the following start scripts from being processed. I changed the boot order to make sure that other services, especially sshd are starting before php-fpm.

Re: PHP-FPM 5.4.5 CentOS 5

perlfan: If you are logged in via SSH and php-fpm hangs when doing a restart of it you can start another SSH session and do a "killall -9 php-fpm"
That will "awaken" your old SSH session that gets in some kind of lock due to php-fpm crashing.

As for the server reboot that was scary, I was under the impression this would only occur when restarting php-fpm
When starting the server fresh no php-fpm processes would be running and thus there would be no problem.
But you say this is not the case?

Re: PHP-FPM 5.4.5 CentOS 5

Likewise, I have never encountered this while initially starting php-fpm. Every indication has been that it's caused by php-fpm not properly shutting down before you start it again. If you're able to replicate it hanging on "service php-fpm start" from a terminal after it's not been started that boot, I'd be interested in what it's doing when it's hung. strace and/or gdb will likely be your friends for that, but they're also not very friendly if you don't know what to do in them. sad

Re: PHP-FPM 5.4.5 CentOS 5

Have you reported this bug to upstream ?

Laptop:  Fedora 38 + rpmfusion + remi (SCL only)
x86_64 builder: Fedora 39 + rpmfusion + remi-test
aarch64 builder: RHEL 9 with EPEL
Hosting Server: CentOS 8 Stream with EPEL, rpmfusion, remi

Re: PHP-FPM 5.4.5 CentOS 5

I haven't reported it upstream as I can't reproduce it reliably enough to get enough information that I'd actually have a bug report that would be acceptable by the PHP folks yet. However, once I manage to figure out how to reproduce it, I intend to try the change noted in https://bugs.php.net/bug.php?id=61231 to see if that corrects the issue. (That looks like a very likely culprit, in my eyes, for buggy process termination.)

My gut says that either a child isn't terminating properly for some reason, or the processes are being terminated in the wrong order somehow.

That's why I'm very interested in that perlfan can apparently reproduce this on a clean startup rather than a restart. If this really is the case and THAT is reproducible, the game is completely changed and any bug report I may submit to upstream may be totally incorrect.

If anyone can reproduce this reliably, perhaps austinh (it looks like it's the same issue, but just in a slightly different guise due to being on a downlevel RHEL?), I'd appreciate information so that I (or maybe Remi? smile ) can reproduce this and it can get reported upstream.

It's entirely possible that there are multiple factors at work here, but the key is finding someone who can reproduce this reliably and can provide information about what they're doing that causes them. As for me, it happens so sporadically (and rarely as of 5.4.6), I have no clue what's going on as I can't poke at it live very well if I can't get it to do anything "useful". hmm

Re: PHP-FPM 5.4.5 CentOS 5

I got this when mailing the Xcache developer about this a few weeks ago:

By using: "DAEMON_COREFILE_LIMIT=unlimited strace -s1024 -f /etc/init.d/php-fpm restart 2>a"
I can see it segv right after the newly started parent process get SIGUSR1 from its child

Reproduced without XCache. Not a XCache problem.

Re: PHP-FPM 5.4.5 CentOS 5

Interesting. You can reproduce this totally reliably? I get no output from the following command:

DAEMON_COREFILE_LIMIT=unlimited strace -s1024 -f /etc/init.d/php-fpm restart 2>&1 | grep -i SEGV

In fact, I have yet to see any issues whenever I try to strace it. sad

Re: PHP-FPM 5.4.5 CentOS 5

yup, I can get it to "fail reliably" this way smile

[root@xeon ~]# php -v
PHP 5.4.6 (cli) (built: Aug 16 2012 14:26:39)
Copyright (c) 1997-2012 The PHP Group
Zend Engine v2.4.0, Copyright (c) 1998-2012 Zend Technologies
    with XCache v2.0.1, Copyright (c) 2005-2012, by mOo


[root@xeon ~]# DAEMON_COREFILE_LIMIT=unlimited strace -s1024 -f /etc/init.d/php-fpm restart 2>&1 | grep -i SEGV
[pid  2956] rt_sigaction(SIGSEGV, {0x7f7ca3e80730, [SEGV], SA_RESTORER|SA_RESTART, 0x7f7caa38a920}, {SIG_DFL, [], 0}, 8) = 0
[pid  2956] rt_sigaction(SIGSEGV, {SIG_DFL, [SEGV], SA_RESTORER|SA_RESTART, 0x7f7caa38a920},  <unfinished ...>
[pid  2956] <... rt_sigaction resumed> {0x7f7ca3e80730, [SEGV], SA_RESTORER|SA_RESTART, 0x7f7caa38a920}, 8) = 0

Re: PHP-FPM 5.4.5 CentOS 5

Can you try to add, in the /etc/sysconfig/php-fpm  file

export ZEND_DONT_UNLOAD_MODULES=1

P.S. with "export" on Enterprise Linux, without "export" on Fedora when systemd is used.

Laptop:  Fedora 38 + rpmfusion + remi (SCL only)
x86_64 builder: Fedora 39 + rpmfusion + remi-test
aarch64 builder: RHEL 9 with EPEL
Hosting Server: CentOS 8 Stream with EPEL, rpmfusion, remi

Re: PHP-FPM 5.4.5 CentOS 5

Patch attached in this bug: https://bugs.php.net/bug.php?id=62886
Also included in the latest snapshot: http://snaps.php.net/php5.4-latest.tar.gz

Re: PHP-FPM 5.4.5 CentOS 5

Thank's for the link (even if I'm already aware)

Laptop:  Fedora 38 + rpmfusion + remi (SCL only)
x86_64 builder: Fedora 39 + rpmfusion + remi-test
aarch64 builder: RHEL 9 with EPEL
Hosting Server: CentOS 8 Stream with EPEL, rpmfusion, remi

Re: PHP-FPM 5.4.5 CentOS 5

php-5.3.17-2 includes this patch.

A new build of php 5.4.7 is planed.

Laptop:  Fedora 38 + rpmfusion + remi (SCL only)
x86_64 builder: Fedora 39 + rpmfusion + remi-test
aarch64 builder: RHEL 9 with EPEL
Hosting Server: CentOS 8 Stream with EPEL, rpmfusion, remi

Re: PHP-FPM 5.4.5 CentOS 5

php-5.4.7-10 includes this patch (and some others)

Laptop:  Fedora 38 + rpmfusion + remi (SCL only)
x86_64 builder: Fedora 39 + rpmfusion + remi-test
aarch64 builder: RHEL 9 with EPEL
Hosting Server: CentOS 8 Stream with EPEL, rpmfusion, remi

Re: PHP-FPM 5.4.5 CentOS 5

Thank you Remi!
My comment wasn't really directed at you, I was not expecting a backported fix smile

Anyway I have tested it on my testserver and I'm no longer able to reproduce the bug!
Will bring it to my live site in the weekend probably.
Thank you so much!