Reply
Is one big query better than many small ones?
Old 03-29-2008, 01:27 PM Is one big query better than many small ones?
Skilled Talker

Posts: 73
Name: Mattias Nordahl
Location: Sweden
I've got a MySQL table for images (with image ID, path etc., not the actual image in a blob). I have a Image class with a function for retrieving an image given an ID as parameter. On some pages there may be anything between 0 and around 30 images.

To the question. How is the page loading time affected if I run one query for each image retrieval, compared to if I were to build a large WHERE clause and retrieve all images at once? Which is the better and how much difference is there?

Thanks
lizciz
lizciz is offline
Reply With Quote
View Public Profile
 
When You Register, These Ads Go Away!
     
Old 03-29-2008, 05:38 PM Re: Is one big query better than many small ones?
joder's Avatar
Flipotron

Posts: 6,350
Name: James
Location: In the ocean.
What I would do is test both. As well as seeing how each loads in the browser, use EXPLAIN to give you information on each one:

EXPLAIN SELECT ...
joder is offline
Reply With Quote
View Public Profile
 
Old 03-30-2008, 01:19 AM Re: Is one big query better than many small ones?
Ultra Talker

Posts: 308
I think there will probably be a common criteria for listing images on a single page. Either they all belong to same album, have a common tag, were added on the same date, etc. If that is the case then using where clause to get them all at once will definitely be efficient. But if you're thinking of building a large where clause by adding comparison to image_id lots of times, well then that would be terribly inefficient and probably not required. Re-think why you need all those images on a single page and i am sure, you'd be able to come up with a common criteria for those images to put in where clause.
__________________
tiny url
dman_2007 is offline
Reply With Quote
View Public Profile Visit dman_2007's homepage!
 
Old 03-30-2008, 06:27 AM Re: Is one big query better than many small ones?
Skilled Talker

Posts: 73
Name: Mattias Nordahl
Location: Sweden
Ok, I see.
For better understanding I'll take an example. I have a page for news (not news you get on the TV, but news regarding the website), which may contain images. Lets say each set of news can have 6 images, and the 5 most recent news will be shown.

The Image table is used for all (uploaded) images on the site, there is a table for the news (table 'News') and then there is a linking table for these two (table 'NewsImages'). How would I best do to retrieve the 5 news together with their respective images?

I'm thinking I could first retrieve the 5 news, and then for each set of news I'll join Images and NewsImages to get the image paths where the news ID equals to the one from the set of news.
I'm guessing there isn't a way to retrieve the all the news togeather with all of it's images in a single query(?).
lizciz is offline
Reply With Quote
View Public Profile
 
Old 03-30-2008, 02:24 PM Re: Is one big query better than many small ones?
joder's Avatar
Flipotron

Posts: 6,350
Name: James
Location: In the ocean.
Using a join.

http://dev.mysql.com/doc/refman/5.0/en/join.html

http://www.tizag.com/mysqlTutorial/mysqlleftjoin.php
joder is offline
Reply With Quote
View Public Profile
 
Old 03-30-2008, 03:24 PM Re: Is one big query better than many small ones?
Skilled Talker

Posts: 73
Name: Mattias Nordahl
Location: Sweden
I'm not sure of what you mean. Is there a way to retrieve all the news and their images in a single query?
lizciz is offline
Reply With Quote
View Public Profile
 
Old 03-30-2008, 04:17 PM Re: Is one big query better than many small ones?
joder's Avatar
Flipotron

Posts: 6,350
Name: James
Location: In the ocean.
Yes. You have to use a join sql statement to join the tables in the query.
joder is offline
Reply With Quote
View Public Profile
 
Old 03-30-2008, 04:33 PM Re: Is one big query better than many small ones?
Skilled Talker

Posts: 73
Name: Mattias Nordahl
Location: Sweden
Ok, I still don't understand how, since I need both the news and the images and each news can have several images. If there was only 1 image I could get a joined table row like

NewsID, NewsAttr1, NewsAttr2, ImageID

But with several images, lets say 4, it would have to look something like

NewsID, NewsAttr1, NewsAttr2, ImageID, ImageID, ImageID, ImageID

Could you perhaps make up an example query?
lizciz is offline
Reply With Quote
View Public Profile
 
Old 03-30-2008, 05:17 PM Re: Is one big query better than many small ones?
chrishirst's Avatar
Super Moderator

Posts: 11,872
Location: Blackpool. UK
http://www.w3schools.com/sql/sql_join.asp

A "joined" recordset would "look" like

NewsID, NewsAttr1, NewsAttr2, ImagePath
NewsID, NewsAttr1, NewsAttr2, ImagePath
NewsID, NewsAttr1, NewsAttr2, ImagePath
NewsID, NewsAttr1, NewsAttr2, ImagePath

Can't write an specific example query without knowing your table structure and the PK / FK relations.

but as a semi non specific example

Code:
SELECT n.newsid,n.newsattr1,n.newsattr2,i.imagepath FROM news AS n LEFT JOIN images AS i ON n.newsid = i.newsid WHERE newsid = ID_to_be_selected;
and if you wanted ALL the rows leave off the WHERE clause

using a LEFT JOIN will retrieve all records from the news table even if there are no matching records from the images table
__________________
Chris. ->> Links are advertising NOT optimising!! <<-
Indifference will be the downfall of mankind, but who cares?
Code Samples | People Counting System
chrishirst is offline
Reply With Quote
View Public Profile Visit chrishirst's homepage!
 
Old 03-30-2008, 09:26 PM Re: Is one big query better than many small ones?
VirtuosiMedia's Avatar
Webmaster Talker

Posts: 603
Quote:
Originally Posted by lizciz View Post
Ok, I still don't understand how, since I need both the news and the images and each news can have several images. If there was only 1 image I could get a joined table row like

NewsID, NewsAttr1, NewsAttr2, ImageID

But with several images, lets say 4, it would have to look something like

NewsID, NewsAttr1, NewsAttr2, ImageID, ImageID, ImageID, ImageID

Could you perhaps make up an example query?
Have you normalized your database?
VirtuosiMedia is offline
Reply With Quote
View Public Profile Visit VirtuosiMedia's homepage!
 
Old 03-31-2008, 01:45 AM Re: Is one big query better than many small ones?
Skilled Talker

Posts: 73
Name: Mattias Nordahl
Location: Sweden
Quote:
Originally Posted by VirtuosiMedia View Post
Have you normalized your database?
Uhm, possibly
I'm still not that good with databases and I don't know all the steps in normalization.

My structure looks like this:

table News
NewsID, Date, Content

table Images
ImageID, ImagePath

table NewsImages
NewsID, ImageID

What I meant with my last post was that if I'm going to select 5 news, and lets say they each have 4 images, the rows would have to "look" like this (right?).

1: NewsID1, Date1, Content1, ImageID1.1, ImageID1.2, ImageID1.3, ImageID1.4
2: NewsID2, Date2, Content2, ImageID2.1, ImageID2.2, ImageID2.3, ImageID2.4
3: NewsID3, Date3, Content3, ImageID3.1, ImageID3.2, ImageID3.3, ImageID3.4
4: NewsID4, Date4, Content4, ImageID4.1, ImageID4.2, ImageID4.3, ImageID4.4
5: NewsID5, Date5, Content5, ImageID5.1, ImageID5.2, ImageID5.3, ImageID5.4
lizciz is offline
Reply With Quote
View Public Profile
 
Old 03-31-2008, 02:54 AM Re: Is one big query better than many small ones?
Ultra Talker

Posts: 308
Quote:
Originally Posted by lizciz View Post
Uhm, possibly
I'm still not that good with databases and I don't know all the steps in normalization.

My structure looks like this:

table News
NewsID, Date, Content

table Images
ImageID, ImagePath

table NewsImages
NewsID, ImageID

What I meant with my last post was that if I'm going to select 5 news, and lets say they each have 4 images, the rows would have to "look" like this (right?).

1: NewsID1, Date1, Content1, ImageID1.1, ImageID1.2, ImageID1.3, ImageID1.4
2: NewsID2, Date2, Content2, ImageID2.1, ImageID2.2, ImageID2.3, ImageID2.4
3: NewsID3, Date3, Content3, ImageID3.1, ImageID3.2, ImageID3.3, ImageID3.4
4: NewsID4, Date4, Content4, ImageID4.1, ImageID4.2, ImageID4.3, ImageID4.4
5: NewsID5, Date5, Content5, ImageID5.1, ImageID5.2, ImageID5.3, ImageID5.4
No if you use the left join, the rows will looks like this:

1: NewsID1, Date1, Content1, ImageID1.1
2: NewsID1, Date1, Content1, ImageID1.2
3: NewsID1, Date1, Content1, ImageID1.3
4: NewsID1, Date1, Content1, ImageID1.4
5: NewsID2, Date2, Content2, ImageID2.1
6: NewsID2, Date2, Content2, ImageID2.2
.................................................. ...........

You'll have to filter the duplicate news content in your application code.
__________________
tiny url
dman_2007 is offline
Reply With Quote
View Public Profile Visit dman_2007's homepage!
 
Old 03-31-2008, 07:00 PM Re: Is one big query better than many small ones?
Skilled Talker

Posts: 73
Name: Mattias Nordahl
Location: Sweden
Ah, I see. Is it just me or does that seem a bit unnecessary? Wouldn't it be eaiser then to first get all the news, and then run a query for each news to get it's images?

Or maybe I'll try both and see which one is faster
lizciz is offline
Reply With Quote
View Public Profile
 
Old 04-01-2008, 06:11 AM Re: Is one big query better than many small ones?
chrishirst's Avatar
Super Moderator

Posts: 11,872
Location: Blackpool. UK
You won't see a lot of difference until you have a LOT of news items.

think about it;

database servers optimise queries "on the fly" to use memory and CPU time efficiently, so a single query pulling all rows required is going to be more efficient than having maybe a thousand or more sequential database trips all using the same query with only the ID changing.

You are thinking small, to design databases and query them successfully and efficiently you have to scale your ideas to HUGE!!!

think in terms of a million+ records to retrieve, then you'll begin to consider what is more efficient.
__________________
Chris. ->> Links are advertising NOT optimising!! <<-
Indifference will be the downfall of mankind, but who cares?
Code Samples | People Counting System
chrishirst is offline
Reply With Quote
View Public Profile Visit chrishirst's homepage!
 
Old 04-01-2008, 04:06 PM Re: Is one big query better than many small ones?
Skilled Talker

Posts: 73
Name: Mattias Nordahl
Location: Sweden
Point taken.
Thanks for your awnsers everyone!
lizciz is offline
Reply With Quote
View Public Profile
 
Old 04-01-2008, 05:56 PM Re: Is one big query better than many small ones?
Learning Newbie's Avatar
Moderator

Latest Blog Post:
What Does This Look Like?
Posts: 4,727
Name: John Alexander
Quote:
Originally Posted by lizciz View Post
Ah, I see. Is it just me or does that seem a bit unnecessary? Wouldn't it be eaiser then to first get all the news, and then run a query for each news to get it's images?

Or maybe I'll try both and see which one is faster
The join will be considerably faster. #1 it's what the database is good at. #2 the database was written in C++ and runs compiled code that's heavily performance optimized, while your PHP code is text being interpreted on the server. Anything that runs in the database will be faster than anything you can write in PHP. That isn't a knock on your coding skills, that's the reality of native machine code and interpreters.
__________________
4 ways to improve the lives of the "bottom billion"

"HEY YOU KIDS GET OFF MY LAWN!" -John McCain
Learning Newbie is offline
Reply With Quote
View Public Profile
 
Old 04-08-2008, 11:51 AM Re: Is one big query better than many small ones?
Novice Talker

Posts: 18
Name: ohsuria
imho..

I ever stored image in sql database.. it save your hard drive space but it make your database server works harder, and it takes your user's browser longer time to process.
then I change the code to create a folder based on the id and store the physical image file.

please cmiiw
ohsuria is offline
Reply With Quote
View Public Profile
 
Reply     « Reply to Is one big query better than many small ones?
 

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off




   
RSS Feed  Feeds: RSS   JS   XML
RSS Feed  Feeds for this forum: RSS   JS   XML