As you might know, I have a server on which I put anything that either needs a pretty stable and good broadband connection or something that should be available 24/7 – things such as my websites and some other servers. Since I already have a server, I have the benefit that I don’t need to pay for a cloud backup service such as Dropbox or OneDrive. Instead, I can simply stuff my server’s hard drive with data until it bursts. For that, I’m using a Nextcloud instance. This thing is awesome – it manages all my events and calendars, contacts, my files, and a lot of other super useful stuff. However, whenever I go into my admin settings, I am greeted with an ominous message: “The PHP memory limit is below the recommended value of 512MB”.
“Okay, easy, I’ll just adapt the php.ini file and the error will be gone!” — that, at least, was my thought over a year ago. For some background: PHP is a programming language that is pretty common in web design: Everything that pertains the backend, i.e. everything computing-intensive which needs to be done on the server is being done in PHP, which then sends back some response. The backend is to be differentiated from the frontend, i.e. everything JavaScript: The stuff that runs and appears on your browser. As PHP runs on my server, it offers a host of configuration options, depending on what I want. For example, I can choose which service I want to use for caching (e.g., Redis or Memcached), how long scripts should run before they’re considered dead, and other stuff. I can adapt those options in a file called php.ini: This file tells my webserver how PHP should run. However, as everything that can be configured, PHP also has default values. Whenever the service starts on my server, it will first set some default values for all configuration values. Afterwards, it will set values from the php.ini-file. And then it’ll start and begin answering requests from people visiting my server.
So when I was greeted with the message I was first a little bit puzzled, but then I realized: Nextcloud is a pretty big service. After all, it has to manage files on the server, synchronize that with all the desktop clients that I have across my devices, and also it offers calendar and contacts syncing, which also has some impact on the required amounts of memory and computing power. And the default values from PHP are meant for light scripts. The default values are perfectly suitable for most production settings, but only if the scripts that are being run are not all too expensive. However, Nextcloud has a few scripts that – due to the sheer amount of data that needs to be processed – could potentially crash if it runs out of memory. So for those big and unusual services, one will need to adapt the initialization configuration. So I opened the corresponding php.ini-file, searched for memory_limit, saw that the line read memory_limit=128M – the default – and changed it to memory_limit=1G (my server has quite a lot of RAM available, so that’s perfectly fine). Alright, I reloaded the affected services, reloaded the Nextcloud admin page … and read “The PHP memory limit is below the recommended value of 512MB”.
What?! So I began digging deeper. I already knew there are actually two php.ini-files: One that is being run whenever someone requests a website, which in my case resides in the folder apache2. And another one that is being run if you run the PHP interpreter from the command line. That one resides in the folder cli. For those of you who are wondering: Yes, you can basically run PHP scripts exactly like you could run Python scripts. Instead of python my_script.py you would simply run php my_script.php. PHP is, like Python, an interpreted language so it works pretty similar to Python. After I had changed both files (since the warning Nextcloud gave me could’ve been caused by any of them), I still got the error.
Now, at that point I thought: “Maybe it’s just a minor bug in Nextcloud. After all, the instance works perfectly well, and doesn’t experience any problems.” And that made sense: After all, my Nextcloud instance is being used by five people, and not hundreds like the commercial installations that are also out there. So it’s pretty unlikely that any of the scripts would actually run out of memory. Still, it would be better if the warning disappeared, because it’s certainly not there for nothing. So I waited, and after every update, I checked the settings page, saw the warning, googled a little bit, but didn’t find anything new.
Yesterday was another update and, in a short break during the summer school I decided to once again investigate. I checked my old friend, the forum thread that contains a lot of complaints by other users about the same warning, and skimmed a few of the posts. As I imagined: Most users fixed the error by updating both php.ini-files. But I already did that – a year ago. So this was a dead end. But I saw some pretty desparate users who recommended just running php -i to output the configuration values as they are actually being used by PHP. I did so and could confirm: The CLI-scripts were given 1GB of memory. But then I realised “Okay, let’s check what the Apache-scripts are getting.” So I created a test file and put the commonly known phpinfo-code in there:
<?php
phpinfo();I opened the file in my browser and searched for memory_limit. And there it was: memory_limit: 128M. WHAT?!
Unbelieving, I checked the Apache-INI and there it read memory_limit=1G. Okay, something fishy is afoot. The phpinfo-function also outputs where the PHP interpreter takes its configuration values from. It told me that it had read the php.ini in the Apache-directory, and also a few config-files from the conf.d-directory. So I checked the conf.d-directory, but: Nothing. These files only contained code that loaded all the required PHP extensions. No memory_limit=128M.
Now I was disturbed where this came from. Then I realized you can also manually override certain values from the configuration per website in the configuration from Apache itself. So I frantically checked each configuration file for every website of mine to check if there was something in there, but: Nothing. A few websites actually did overwrite a few PHP values, but no memory_limit. Next, I found that there is a file called .user.ini in my Nextcloud directory. That can also be used to override some PHP values. So I opened it, and: Again, nothing.
At this point, I was really disturbed. My global configuration clearly stated that each script could have 1GB of memory, but both Nextcloud and phpinfo said “No, it’s in fact 128MB, not 1GB”. At this point I was only shy of believing in ghosts. So I carefully read the forum thread again. And then this comment caught my attention:
Nevermind… Typo in the php.ini line 2
][PHP] should have been [PHP]
“Wait a moment …” I was pretty sure that this couldn’t really be the cause, but anyway, I was out of options, so why not open the php.ini yet another time? And there it was:
<<[PHP]
...There was a <<[PHP] at the beginning of the file. It should’ve been just [PHP]. Obviously I never changed that line, because why should I? The php.ini-file is extremely well documented, and all config options are already in there so all I ever did was either comment out or uncomment a few values, and change the defaults of some values. How did it get there? I think it might’ve been me accidentally hitting the <-key twice after opening the file at some point (since the cursor begins at the beginning of the file, i.e. immediately before [PHP]). Nevertheless: It’s an INI-file, and the INI-format is both very old and not very clever. It basically just allows you to define config-sections, delimited by these square brackets, and then configuration values inside these sections, in the form key=value. Today, configuration is mostly done using either dotenv-files (which look a little bit like INI-files, but not quite), or YAML- and TOML-files, the latter of which are being used by, e.g., Rust, and which are basically an improved version of INI-files. When there’s an error in these, you will notice. Either because you accidentally didn’t produce valid YAML/TOML or because the environment variables look off and your software crashes.
But remember what I said at the beginning? PHP would define some sane default values in the code, and then overwrite those with values from the INI-file. So whenever it read in my php.ini, it would look for a section called PHP. Since INI requires those sections to be the only thing on the line, the line <<[PHP] would not be a valid section and be just ignored. So, for the PHP interpreter, there was no PHP-section in the file. And, as a result, PHP would stick to its internal defaults. Apparently, there’s no sanity check in the PHP binary that tells me “Hey, I’ve just read a php.ini-file, but I couldn’t find a section PHP – there might be something wrong with your file!” Instead, it just silently ignores almost 2,000 lines of configuration and does … nothing.
After I removed the two less-than characters, saved the file and restarted the webserver, I checked the Nextcloud settings page, and, lo and behold: The error was gone!
For one year, I questioned my sanity because PHP did not indicate to me that there might be something off in the INI-file. I now feel like the stupidest person on the planet and I wanted to be swallowed by the earth.
And then the break was over and the summer school continued. I was quieter than before the break. But at least, no more errors.
The lesson learned? Even if you know something for almost 15 years, you are still not safe from making rookie-mistakes. Let us face it: If you don’t actively develop some software yourself, you will make such errors, because of so many assumptions and individual experiences which are baked into applications. No one is safe from it, and sometimes a few sanity checks can rescue your own sanity from the insanity of the software we use. So, remember kids: People will make mistakes, and if you have a file that you expect to contain a few things, check for them. Don’t just quietly ignore that. Give out a warning. Tell the people “Hey, there might be something off!” instead of just saying “Nah, it’s fine. I’ll just recover and continue, despite the errors!”
