Slow loading
- Thread starter Momo
- Start date
- Latest activity Latest activity:
I've said this before but all X links freeze the page for me. I have to either find a part of the page in between the links that I can scroll with or I have to back out if the thread and come back in again.
I have noticed slow loading pages too but I just put it down to my WiFi acting up.
I have noticed slow loading pages too but I just put it down to my WiFi acting up.
I had it the other day but yeah in general twitter posts make it hard to scroll
I don't see the problem as a user but I mainly use a browser (Mac or iPad). The server loads are fine currently and I see nothing 'exciting' in the logs.
We did have a problem on Friday but that was due to excessive crawler activity which I think I've managed to block/mitigate.
Happy to have a more extensive look if you can give me something a bit more concrete to look for
We did have a problem on Friday but that was due to excessive crawler activity which I think I've managed to block/mitigate.
Happy to have a more extensive look if you can give me something a bit more concrete to look for
OK, I'm out of my depth here and we do need Jono or another website/PHP god to step in. I've increased the PHP pools and threads but beyond that I'm not knowledgeable enough to do much else. Response times look good from curling pages so then the focus goes on to local render
[edit] ChatGPT has suggested I turn on slowlog which I've done too
@6TimesaRed ⭐ ?
[edit] ChatGPT has suggested I turn on slowlog which I've done too
@6TimesaRed ⭐ ?
Can you report back into this thread with any findings positive or otherwise?
@Momo @Cerberus @SummerOnions
@Momo @Cerberus @SummerOnions
To trace I need the time you had the problem and the specific post in thread
I just had it now on this Twitter post https://www.sixcrazyminutes.com/threads/great-goals-25-26.218665/page-2#post-2495626
I think it's not specific to this site though, it's an X issue.
I think it's not specific to this site though, it's an X issue.
That reacts instantly on my phone just now (20:00) and curling the URL gets a quick response too
Curl from my Mac to SCM:
DNS: 0.002914s
Connect: 0.027327s
TLS: 0.292499s
TTFB: 0.486719s
Total: 0.536444s
X direct embed:
Total: 0.275067s
SCM embedding is adding .25s to the twitter native time which is pretty decent
Yep, it impacts pretty much every single X post, just not every time. Something to do with the first time they're loaded, or maybe the nth time they're loaded, not exactly sure. I don't think it impacts any other embeds like Instagram.
I get that issue all the time with twitter links. Wait 15-20 seconds to load up. I put it down to not having it installed in my phone or logged in.
I do have the Twitter app, although I never use it.
OK guys. I've spent a fair bit on tracing and think I get this. For clarity, here’s how the process works.
When a post containing an X link is created, the URL is stored and the post is made live. The first time the post is rendered (which is almost always immediately after posting), XenForo retrieves the embed data from https://publish.twitter.com/oembed.
The returned HTML is then stored in our database and cached. That means under normal circumstances there is only one server-side call to X per post (unless embed caches are manually cleared, which I occasionally do during maintenance — for example I did this on Sunday when resolving the earlier issue). So in practice, the original poster is almost always the one who triggers that initial retrieval, and every subsequent viewer is served the cached embed HTML directly from our database.
When a user 'looks' at the post the HTML content is retrieved from the cache. Almost all X posts we embed have embedded media and needs rendering. This is done client side by the browser or a Progressive web app (PWA). Android phone PWA is Chrome (based on Blink) and iOS is generally Safari (all based on iOS webkit).
The PWA loads the page, XenForo provides the cached embed HTML, and X’s widgets.js script then executes in the browser. That script transforms the embed markup and loads any associated media from X’s CDN. I can't work out if the PWA caches the media or not, the important thing here is that the forum and our server is not involved.
So the heavy lifting is performed on the device (which becomes more acute on a mobile), not on the forum server. Any delay at that stage is dependent on the user’s device, browser engine, and connection — not the forum infrastructure. There are loads of posts on Reddit and elsewhere of brewers having X rendering problems, some with simple solutions to the localised issue, like a plugin needed to be removed.
... and just to be 100% clear. There is no dependancy on having the X app installed on the phone. The underlying engine, either webkit or Blink on a phone, deals with all the traffic and rendering without recourse to the app whether there or not.
When a post containing an X link is created, the URL is stored and the post is made live. The first time the post is rendered (which is almost always immediately after posting), XenForo retrieves the embed data from https://publish.twitter.com/oembed.
The returned HTML is then stored in our database and cached. That means under normal circumstances there is only one server-side call to X per post (unless embed caches are manually cleared, which I occasionally do during maintenance — for example I did this on Sunday when resolving the earlier issue). So in practice, the original poster is almost always the one who triggers that initial retrieval, and every subsequent viewer is served the cached embed HTML directly from our database.
When a user 'looks' at the post the HTML content is retrieved from the cache. Almost all X posts we embed have embedded media and needs rendering. This is done client side by the browser or a Progressive web app (PWA). Android phone PWA is Chrome (based on Blink) and iOS is generally Safari (all based on iOS webkit).
The PWA loads the page, XenForo provides the cached embed HTML, and X’s widgets.js script then executes in the browser. That script transforms the embed markup and loads any associated media from X’s CDN. I can't work out if the PWA caches the media or not, the important thing here is that the forum and our server is not involved.
So the heavy lifting is performed on the device (which becomes more acute on a mobile), not on the forum server. Any delay at that stage is dependent on the user’s device, browser engine, and connection — not the forum infrastructure. There are loads of posts on Reddit and elsewhere of brewers having X rendering problems, some with simple solutions to the localised issue, like a plugin needed to be removed.
... and just to be 100% clear. There is no dependancy on having the X app installed on the phone. The underlying engine, either webkit or Blink on a phone, deals with all the traffic and rendering without recourse to the app whether there or not.
I got ChatGPT to munge through my thinking plus review the apache and XF logs:
This is excellent data — and it pretty much answers the “is this XenForo/PHP slow or is it X/media?” question.
1) Your core thread pages are not slow
Your slowest /threads/... GET in that sample is 0.734s, then 0.663s, then everything drops into the 0.2–0.3s range.
That means:
XF page generation (PHP + MariaDB + ES + template rendering) is behaving fine.
2) The real server-side slowness is
You’ve got:
This tells us:
So for users: “the forum is slow” can be true from their point of view, but the load is dominated by external image/media hosts, not your VPS.
Those are often slow / redirect-y / rate-limited / flaky.
That’s exactly the kind of stuff that makes mobile users think “that page is hanging”.
This is excellent data — and it pretty much answers the “is this XenForo/PHP slow or is it X/media?” question.
1) Your core thread pages are not slow
Your slowest /threads/... GET in that sample is 0.734s, then 0.663s, then everything drops into the 0.2–0.3s range.
That means:
XF page generation (PHP + MariaDB + ES + template rendering) is behaving fine.
- Any “feels slow” experience is almost certainly after HTML arrives (client-side rendering + external assets/media), or it’s image proxy calls dragging the overall page load “complete”.
2) The real server-side slowness is
proxy.php
(image proxy)
You’ve got:
- 6.1s proxy fetch from a WordPress-hosted image (HTTP, not HTTPS)
- Then a bunch of 0.3–1.2s proxy fetches from The Athletic / postimg / tapatalk / etc.
- You do have at least one pbs.twimg.com proxy fetch at ~0.18s in that window.
- 2.7s proxy fetch from Tenor (GIF)
This tells us:
- The big stalls are mostly “random remote hosts are slow” rather than XenForo being slow.
- When an embed includes a preview image (or the post includes inline images), and image proxy is enabled, the browser waits on your server which is waiting on the remote host.
So for users: “the forum is slow” can be true from their point of view, but the load is dominated by external image/media hosts, not your VPS.
3) Your “decode” output confirms you’re proxying a ton of ancient HTTP junk
Your host list shows lots of old http://... images, weird google thumbnail URLs, etc.Those are often slow / redirect-y / rate-limited / flaky.
That’s exactly the kind of stuff that makes mobile users think “that page is hanging”.
I would have thought if it's once only it'd be done at the time of the original post being done, not when someone else loads it.
Correct. THat's what I was trying to say. The poster creates the post in the editor (which doesn't do anything at that point) and clicks Post. This triggers a reload of the page and it would be astonishing if anyother user read the page before the reload happened.
My point is it is not the act of posting that created the retrieval/caching, it is the first view which is 99.9999% going to be the page reload of the poster.
Make sense? I was trying to be as accurate as possible
Yeah, it does, and how I'd guess it should work.
But I don't think that it does work like that, when I see an issue it's not from one I've posted, and pretty unlikely I've timed it so I beat them to it, that often anyway.
But I don't think that it does work like that, when I see an issue it's not from one I've posted, and pretty unlikely I've timed it so I beat them to it, that often anyway.
Similar threads
- Locked
- Locked