PistonHeads Performance and Issues - Latest Update 09/05/12
Discussion
To try and keep everyone up to date with the current performance problems and any issues with PistonHeads the team will keep this post up to date with the latest update so it isn't lost inside various threads.
So, latest news from the bridge:
We have had a switch failure on our network serving the traffic that resulted in outages on Friday, we are working with our hosting company to solve these issues and sort out the current throttling we are seeing on the site (which is causing the slow page loads instead of time outs) - hopefully will be able to feed back the issues with this soon.
The mobile site works by scraping the main site and then serving the content to the specific mobile. It has an inbuilt time out of 4 seconds, so if it doesn't get a response from the main site within that time it gives the time out message you see on occasion - increasing the main site performance as above will solve these issues.
The slowness in past weeks have been down to amount of data and number of users. We are still growing and have been constantly for the last 5 years, with a record number of users last month. This has been a constant challenge to keep the hardware and software for the whole site maintainable for the future. So we started to rebuild the classifieds section as part of this with a massive investment 18 months ago - as this site is huge and we have a lot of things to do this is still in progress, but will future proof the site looking forward.
With the planned launch of the classifieds last December we had hoped that we would have a lot of traffic off the old platform, but that failed to happen and we are suffering at the moment as we roll out the newly rebuilt classifieds in a softer approach.
We have done some further tuning on the data and way some of the legacy code works to help with the performance issues, and will continue to do this. All the silver bullet coding stuff has been done to the code base, and thus the classic ASP site is running as good as you can get the code.
It is frustrating for all of us in the office, as well as you guys out there, so we feel your pain as we want to use the site as much as you do. As the beta traffic increases, further tuning of the dataset and a few tweaks to the way we handle the data in the site will increase performance over the coming week. So I am really sorry for all the problems and the team are working hard on sorting this.
Update 09/05/12 @ 16:00
We have highlighted the problem of the slow response from the servers - It was due to a front end proxy hitting disk too much and causing a slow response, as we have three of these servers it was pot luck which one you hit and thus throttled the performance of the site.
This has now been rectified so should improve site response and mobile response.
Update 16/05/12 @ 13:05
We are currently working with our hosting provider to find a suitable time to put in place the permanent fix to the switch failure bug we have in the system. This will be a five hour downtime and so choosing a suitable time is the challenge at the moment as it effects the whole infrastructure. Best bet is an early Sunday morning between 7am and 12pm - will try and not do it when there is a popular event like F1.
This fix will enable us to have full access to the servers again, and means our monitoring system will be able to alert us of any problems before they happen.
We are also looking at the performance of the database, as this is causing the slow down in the evenings - hopefully have an update on this later this week.
Update 17/05/12 @ 15:55
We have found a solution to get the network fix in place tonight much quicker, so site will be down for around 10 minutes at 23:30. This will bring us back to normal, and our monitoring will be back for tomorrow and allow us to start work on the further database tuning we need to do.
So, latest news from the bridge:
We have had a switch failure on our network serving the traffic that resulted in outages on Friday, we are working with our hosting company to solve these issues and sort out the current throttling we are seeing on the site (which is causing the slow page loads instead of time outs) - hopefully will be able to feed back the issues with this soon.
The mobile site works by scraping the main site and then serving the content to the specific mobile. It has an inbuilt time out of 4 seconds, so if it doesn't get a response from the main site within that time it gives the time out message you see on occasion - increasing the main site performance as above will solve these issues.
The slowness in past weeks have been down to amount of data and number of users. We are still growing and have been constantly for the last 5 years, with a record number of users last month. This has been a constant challenge to keep the hardware and software for the whole site maintainable for the future. So we started to rebuild the classifieds section as part of this with a massive investment 18 months ago - as this site is huge and we have a lot of things to do this is still in progress, but will future proof the site looking forward.
With the planned launch of the classifieds last December we had hoped that we would have a lot of traffic off the old platform, but that failed to happen and we are suffering at the moment as we roll out the newly rebuilt classifieds in a softer approach.
We have done some further tuning on the data and way some of the legacy code works to help with the performance issues, and will continue to do this. All the silver bullet coding stuff has been done to the code base, and thus the classic ASP site is running as good as you can get the code.
It is frustrating for all of us in the office, as well as you guys out there, so we feel your pain as we want to use the site as much as you do. As the beta traffic increases, further tuning of the dataset and a few tweaks to the way we handle the data in the site will increase performance over the coming week. So I am really sorry for all the problems and the team are working hard on sorting this.
Update 09/05/12 @ 16:00
We have highlighted the problem of the slow response from the servers - It was due to a front end proxy hitting disk too much and causing a slow response, as we have three of these servers it was pot luck which one you hit and thus throttled the performance of the site.
This has now been rectified so should improve site response and mobile response.
Update 16/05/12 @ 13:05
We are currently working with our hosting provider to find a suitable time to put in place the permanent fix to the switch failure bug we have in the system. This will be a five hour downtime and so choosing a suitable time is the challenge at the moment as it effects the whole infrastructure. Best bet is an early Sunday morning between 7am and 12pm - will try and not do it when there is a popular event like F1.
This fix will enable us to have full access to the servers again, and means our monitoring system will be able to alert us of any problems before they happen.
We are also looking at the performance of the database, as this is causing the slow down in the evenings - hopefully have an update on this later this week.
Update 17/05/12 @ 15:55
We have found a solution to get the network fix in place tonight much quicker, so site will be down for around 10 minutes at 23:30. This will bring us back to normal, and our monitoring will be back for tomorrow and allow us to start work on the further database tuning we need to do.
Edited by RacingPete on Thursday 17th May 15:54
Gassing Station | Website Feedback | Top of Page | What's New | My Stuff