No, I woke up, saw the alert, bounced httpd, and had to run out the door.
It's a reasonable guess that it's the same memory issue as before. We're currently bouncing apache in cron every-other-hour, which seems to mostly keep us up, but occasionally not, so my guess is that's RIGHT at the threshold.
I don't have the bandwidth the go spelunking - the httpd.conf adjustments I made in... November or whatever that was seemed to help, but obviously they didn't solve everything. I still suspect that Drupal is causing connections to be held open somewhere, but that's pure speculation.
We have the data, between the apache_status.log and DD to figure it out, I'd just need time to do it, and I don't have the cycles right now.
On 1/7/24 17:11, Ilan Rabinovitch wrote:
Looks like we had another outage today. Any idea whats up?
On Mon, Dec 25, 2023 at 1:17 AM Ilan Rabinovitch <ilan@linuxfests.org mailto:ilan@linuxfests.org> wrote:
Looks like another outage about 2 hours ago. On Sun, Dec 24, 2023 at 8:45 PM Ilan Rabinovitch <ilan@linuxfests.org <mailto:ilan@linuxfests.org>> wrote: website was down again.. looks like for about an hour. ive just restarted httpd. On Sat, Oct 21, 2023 at 5:00 PM Phil Dibowitz <phil@ipom.com <mailto:phil@ipom.com>> wrote: Logs look good. https://github.com/socallinuxexpo/scale-chef/pull/302 <https://github.com/socallinuxexpo/scale-chef/pull/302> drops restarts to every 2 hours. Merged. On 10/21/23 13:47, Phil Dibowitz wrote: > I dropped it to less often a few days ago, I haven't yet looked at the > logs to see what the status of httpd is at those times. My plan is to do > that, then drop it to less often, and rinse and repeat. > > Sorry new job is keeping me super busy. > > On 10/21/23 08:25, Ilan Rabinovitch wrote: >> Are we able to remove the cron job that's restarting httpd? >> >> On Sun, Oct 8, 2023 at 12:48 AM Phil Dibowitz <phil@ipom.com <mailto:phil@ipom.com>> wrote: >>> >>> On 10/7/23 14:30, Phillip Smith wrote: >>>> Yes, let's schedule something. I'm available tomorrow morning and >>>> Tuesday evening. >>> >>> What can I provide you in the mean time? I thought I saw an email from >>> you saying you wanted Datadog access, but I can't seem to find it. >>> >>> Davide may have time Tuesday, I will have time Wed evening. I'm out of >>> town until Monday and have plans Mon/Tues evening. >>> >>> - Phil >>> >>> >>> _______________________________________________ >>> scale-infra mailing list >>> scale-infra@lists.linuxfests.org <mailto:scale-infra@lists.linuxfests.org> >>> https://lists.linuxfests.org/cgi-bin/mailman/listinfo/scale-infra <https://lists.linuxfests.org/cgi-bin/mailman/listinfo/scale-infra> >> _______________________________________________ >> scale-infra mailing list >> scale-infra@lists.linuxfests.org <mailto:scale-infra@lists.linuxfests.org> >> https://lists.linuxfests.org/cgi-bin/mailman/listinfo/scale-infra <https://lists.linuxfests.org/cgi-bin/mailman/listinfo/scale-infra> > -- Phil Dibowitz phil@ipom.com <mailto:phil@ipom.com> Open Source software and tech docs Insanity Palace of Metallica http://www.phildev.net/ <http://www.phildev.net/> http://www.ipom.com/ <http://www.ipom.com/> "Be who you are and say what you feel, because those who mind don't matter and those who matter don't mind." - Dr. Seuss _______________________________________________ scale-infra mailing list scale-infra@lists.linuxfests.org <mailto:scale-infra@lists.linuxfests.org> https://lists.linuxfests.org/cgi-bin/mailman/listinfo/scale-infra <https://lists.linuxfests.org/cgi-bin/mailman/listinfo/scale-infra>
scale-infra mailing list scale-infra@lists.linuxfests.org https://lists.linuxfests.org/cgi-bin/mailman/listinfo/scale-infra