Database look-up : POST or GET difference?
Discussion
First, you'll have to forgive me as I may have the terminology wrong 
. For a database look-up page, what are the advantages and disadvantages of having the data show in the URL ? If I understand correctly, POST doesn't do this and the page URL stays the same, but GET puts it in the URL. Some experience of using another site which has the data in the URL strongly suggests that this method makes the site rank highly in search engines which is obviously a good thing, but are there any disadvantages?
. For a database look-up page, what are the advantages and disadvantages of having the data show in the URL ? If I understand correctly, POST doesn't do this and the page URL stays the same, but GET puts it in the URL. Some experience of using another site which has the data in the URL strongly suggests that this method makes the site rank highly in search engines which is obviously a good thing, but are there any disadvantages?Http Verbs are used for different things.
POST = Create Data (Most commonly)
GET = Read Data
https://restfulapi.net/http-methods/
POST = Create Data (Most commonly)
GET = Read Data
https://restfulapi.net/http-methods/
Using GET to do the lookup page makes it bookmarkable and thus the results page can be indexed in a search engine. Not sure that it would make the page appear any higher in the rankings though as its quite complex how they are ranked. If the results of the database lookup correlate with the search terms then it could work to increase the ranking
Passing data via a GET request is done by the query string, Look in the address bar as you browse this forum you will see a ? after the page name and then a lot of variables=values with an & between them, when doing a POST the variable=values pairs wont be visible directly in the browser. The disadvantages of this method are firstly the query string has a character limit of 2048, whereas POST is unlimited within reason. The 2nd would be if you are passing sensitive information. You wouldn't pass a password via query string as a) above it can be bookmarked and thus is stored on the users device in plaintext and also would be viewable to any busy body looking over the shoulder of a user.
What are you trying to achieve and thus why do you care how the data is passed?
Passing data via a GET request is done by the query string, Look in the address bar as you browse this forum you will see a ? after the page name and then a lot of variables=values with an & between them, when doing a POST the variable=values pairs wont be visible directly in the browser. The disadvantages of this method are firstly the query string has a character limit of 2048, whereas POST is unlimited within reason. The 2nd would be if you are passing sensitive information. You wouldn't pass a password via query string as a) above it can be bookmarked and thus is stored on the users device in plaintext and also would be viewable to any busy body looking over the shoulder of a user.
What are you trying to achieve and thus why do you care how the data is passed?
Thanks all.
Scabutz, it's not sensitive data. However, the data is unique. Imagine something like car registration numbers (it's not that but it's a decent example) and people would be searching the web for those exact matches. Other established websites do eventually list this data as it's me and another guy who send it to them
, but by putting the data out there ourselves we have identified a good way of generating revenue from a related service if we can get the traffic to our site.
As a new site it will take considerable time to become widely known hence why I've been thinking about using GET and having the unique data in the URL (it's either a 6 char set or max 10 char set, depending on what the user searches for).
As a side note, I have a phpBB forum attached albeit currently unused as the site isn't even built yet. 5 days ago I dropped some unique and completely made up 'codes' in a couple of forum posts and a search for them on google, yahoo and bing all have the forum posts indexed with the codes. If the 'codes' in a GET URL index as quickly, I'd be a very happy chappy. Seems from what you've said there aren't any downsides that would cause me a problem.
Scabutz, it's not sensitive data. However, the data is unique. Imagine something like car registration numbers (it's not that but it's a decent example) and people would be searching the web for those exact matches. Other established websites do eventually list this data as it's me and another guy who send it to them
, but by putting the data out there ourselves we have identified a good way of generating revenue from a related service if we can get the traffic to our site.As a new site it will take considerable time to become widely known hence why I've been thinking about using GET and having the unique data in the URL (it's either a 6 char set or max 10 char set, depending on what the user searches for).
As a side note, I have a phpBB forum attached albeit currently unused as the site isn't even built yet. 5 days ago I dropped some unique and completely made up 'codes' in a couple of forum posts and a search for them on google, yahoo and bing all have the forum posts indexed with the codes. If the 'codes' in a GET URL index as quickly, I'd be a very happy chappy.
Seems from what you've said there aren't any downsides that would cause me a problem.If you have the search terms in the URL then it's much easier for a third party to run automation across your dataset.
If you're talking about data relating to a car registration and I create a list of all possible registrations I only have to ask Excel nicely and it'll go and get all your data.
Is that something that would be a problem for you?
If you're talking about data relating to a car registration and I create a list of all possible registrations I only have to ask Excel nicely and it'll go and get all your data.
Is that something that would be a problem for you?
paulrockliffe said:
If you have the search terms in the URL then it's much easier for a third party to run automation across your dataset.
If you're talking about data relating to a car registration and I create a list of all possible registrations I only have to ask Excel nicely and it'll go and get all your data.
Is that something that would be a problem for you?
It would be as easy to screen scrape a set of results using post or get - slightly more complicated code to write, but not really difficult, if data is available to someone without logging in, it can be scraped...If you're talking about data relating to a car registration and I create a list of all possible registrations I only have to ask Excel nicely and it'll go and get all your data.
Is that something that would be a problem for you?
I'm not in the SEO game at all but I was under the impression that querystrings were a bit old hat? I thought Google preferred URLs that looked like they were directories, e.g.
https://www.somesite.com/search.php?id=wibble&page=2
..would be:
https://www.somesite.com/search/id/wibble/page/2
I've tended to see querystrings only really used now for filtering or sorting, i.e. stuff that you wouldn't want Google to index anyway (as it's duplicate content).
As said above also exposing the framework of your search mechanism via GET requests also means it can be scraped by quite simple tools, e.g. like Excel, unless you take steps to try and prevent that, which is additional workload.
https://www.somesite.com/search.php?id=wibble&page=2
..would be:
https://www.somesite.com/search/id/wibble/page/2
I've tended to see querystrings only really used now for filtering or sorting, i.e. stuff that you wouldn't want Google to index anyway (as it's duplicate content).
As said above also exposing the framework of your search mechanism via GET requests also means it can be scraped by quite simple tools, e.g. like Excel, unless you take steps to try and prevent that, which is additional workload.
Paul, that could be a concern. I had planned to implement a CAPTCHA kinda thing upon detecting an unreasonable amount of requests from the same source within a short period. Admittedly I'm not clued up on that stuff but I imagine there will be tools to prevent bot scraping, or at least make the process so annoying for them that they give up.
I can't remember what site now as it was a lot of years ago, but I remember keying in a number of requests in fairly quick succession to see if they had the data in their datbase and after about 20 I got some sort of 'anti-flood' message citing excessive usage and to come back later. It worked again after a minute or so break. Technology has probably moved on since then and there are better ways?
But as Kirk says, surely they could run a bot program to do same via the POST search box?
ETA: the search needs to be public and free access for quick look-ups. No username creation and sign-up to access the site as that will drive most people away.
I can't remember what site now as it was a lot of years ago, but I remember keying in a number of requests in fairly quick succession to see if they had the data in their datbase and after about 20 I got some sort of 'anti-flood' message citing excessive usage and to come back later. It worked again after a minute or so break. Technology has probably moved on since then and there are better ways?
But as Kirk says, surely they could run a bot program to do same via the POST search box?
ETA: the search needs to be public and free access for quick look-ups. No username creation and sign-up to access the site as that will drive most people away.
Edited by PH4555 on Wednesday 18th August 09:54
Durzel said:
You can implement an Google CAPTCHA that is invisible nowadays, so it ought not to be any burden on a genuine user at all, but would (should?) stop the more casual scraping of the website.
Good to know. That's the kind of thing I had in mind, although hopefully not those highly annoying "select all the pictures containing boats" type . I'm sure the most determined 'hackers' would find a way, but it's just something to stop the opportunists who may think they can extract my entire database by running an Excel file against it. Finding the right balance without annoying the f
k out of genuine users by filling their screen with CAPTCHA stuff will require some fine-tuning.Apologies if you're already aware and this is irrelevant to your original post/get question, but.... If you're accepting user input, and using this to run database queries you really need to be escaping and filtering the input prior to it being used.
SQL Injection attacks are quite simple to perform, and it's not something ReCaptcha will defend against.
Depending on your application layer, most frameworks/apps will probably have some built in protection, but always worth keeping in mind.
If storing user input, you want to avoid them sending anything harmful - Where I work, one of the devs built an app where I was able to include a script in the name fields which automatically sent back the admin session info when the admin page was loaded. This allowed me to fake an admin session and see all the data (Also due to a badly configured web server)
SQL Injection attacks are quite simple to perform, and it's not something ReCaptcha will defend against.
Depending on your application layer, most frameworks/apps will probably have some built in protection, but always worth keeping in mind.
If storing user input, you want to avoid them sending anything harmful - Where I work, one of the devs built an app where I was able to include a script in the name fields which automatically sent back the admin session info when the admin page was loaded. This allowed me to fake an admin session and see all the data (Also due to a badly configured web server)
Gassing Station | Computers, Gadgets & Stuff | Top of Page | What's New | My Stuff