Reply
problem to handle a large amount of data
Old 07-24-2008, 01:46 AM problem to handle a large amount of data
Junior Talker

Posts: 1
hi frds,
I am working with .net envirement with c#. I want to take a table which contain 10 million data or above with the help of dataset. but my application unable to handle due to large amount.

help required
jainendra is offline
Reply With Quote
View Public Profile
 
When You Register, These Ads Go Away!
Old 07-24-2008, 03:56 AM Re: problem to handle a large amount of data
JeremyMiller's Avatar
Full-Time TeraTasker

Posts: 984
Name: Jeremy Miller
Location: Reno, NV
Others who know C# will provide better assistance, but if you're having memory problems, then try to load only the maximal subset of rows of information you need at any one time. For instance, if the user can only see 25 rows of data, then having 10 million in memory isn't helpful. This can complicate queries, but I know that C# has some DB interfaces and you'll definitely do best to use one of them.

Hope that helps a bit. I'm sure others will come in with more pointed advice.
__________________
Jeremy Miller - TeraTask Technologies, LLC
Content Farmer - Automated Posting for Content & Blog Sites
JeremyMiller is offline
Reply With Quote
View Public Profile Visit JeremyMiller's homepage!
 
Old 07-24-2008, 02:31 PM Re: problem to handle a large amount of data
Learning Newbie's Avatar
Moderator

Posts: 5,199
Name: John Alexander
Jeremy is basically right, although C# and SQL interaction isn't really something that comes into play with the answer here. They're cleanly separated, and should be.

There's absolutely no reason you should need 10,000,000 rows in a DataSet, especially in a web application. You know about page life cycle, right? Only request the data you need. The database is responsible for the storage and exposure of data - your application is responsible for whatever processing it does. It should request only the data that's necessary. Precaching the entire database will work against you - as you're seeing.

I can't say all that much more about how to achieve that, without knowing the details of your application.
__________________
4 ways to improve the lives of the "bottom billion"

"HEY YOU KIDS GET OFF MY LAWN!" -John McCain
Learning Newbie is offline
Reply With Quote
View Public Profile
 
Reply     « Reply to problem to handle a large amount of data
 

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off




   
RSS Feed  Feeds: RSS   JS   XML
RSS Feed  Feeds for this forum: RSS   JS   XML

 


Page generated in 0.11708 seconds with 12 queries