0

I did some profiling on a program that I'm running and the thing that takes the longest time is getting the results from the linq query:

var Results = 
    from a in Table
    group a by a.Value into b
    select new {Group = b};

foreach(var Result in Results)
{
    //Do calcs
}

Any ideas on how I can speed this up?

sooprise
  • 22,657
  • 67
  • 188
  • 276
  • 2
    I don't think there is enough info here. What is Table? Is it making database calls? What is taking a long time? Just the foreach loop? Or what is in the foreach? – Bryan Sep 17 '10 at 20:20
  • 1
    Looks like `select new {Group = b}` could be just `select b` but that won't speed it up. – H H Sep 17 '10 at 20:21
  • Grouping can be an expensive operation. If Table is in a database, does it have an index on Value? – H H Sep 17 '10 at 20:23
  • Table is a table in an SQL database, and yes it does have an index on Value. Does having an index have a significant effect on performance? – sooprise Sep 17 '10 at 20:25
  • yes, but in a good way (when fetching) – Kirk Woll Sep 17 '10 at 20:29

3 Answers3

2

I think you are confusing the query object with the results of that query. Your first variable doesn't contain the results, it contains a query object. At this point the query hasn't been executed. The execution is delayed until you actually need the results, and in your example this is done when you iterate in the foreach loop. This is why the first statement executes quickly but the iteration is slow.

If you want to store the results of the query in results so that they are already calculated by the time you start the foreach loop then add a call to ToList().

var results = 
    (from a in Table
     group a by a.Value into b
     select new {Group = b}).ToList();
Mark Byers
  • 811,555
  • 193
  • 1,581
  • 1,452
  • But would that make it (the whole of the posted code) any faster? – H H Sep 17 '10 at 20:22
  • 1
    I doubt it, but to be sure you should measure. What it will do though is take the execution cost up-front so that the iteration is faster. It seems to me that the OP's concern was more that the iteration was surprisingly slow rather than the total runtime of the code being slow. If you're not sure what is going on behind the scenes then you might expect the first line to be the slow one and the loop to be fast. – Mark Byers Sep 17 '10 at 20:31
  • 1
    I has a list with 128000 items. Had it group by productid. Made a Where on the key and ran though that. It took 0.10 sec. After doing a ToList on the IEnumerable> the time was 0.0002 sec. Dont know that is happing behide the screen. But it makes a big diffents. – mimo Sep 18 '13 at 11:31
0

If you're using .NET 4, have a look at P-LINQ or Parallel ForEach loops. That should dramatically increase performance.

Can't really tell based on the info given, but it may be a SQL query taking too long?

If it's the ForEach loop that's really causing the bottleneck, then the Parallel ForEach will be your best bet.

Steve Danner
  • 21,818
  • 7
  • 41
  • 51
  • If it's I/O bound (database) a few extra threads won't help much. I think it depends (on unavailable details). – H H Sep 17 '10 at 20:25
  • That will only have a potential improvement for LINQ to Objects, but if this is using an IQueryable, it won't help (much), since the bottleneck is probably in the backend. – Reed Copsey Sep 17 '10 at 20:26
  • @Reed, yeah it's like Henk said, it's based completely on unavailable details. If the Calcs being done in the foreach loop are the issue, this would help, or even if the LINQ statement is against an object collection, but we don't know from the info given. – Steve Danner Sep 17 '10 at 20:30
0

I have had the same problem with only 2000 Sqlite records (indexed) used for an MSChart, apparently this is caused by the LINQ group statement querying all records.

The only solution I found was to go back to native SQL and the chart rendered instantaneously rather than 2 seconds using LINQ with SQlite.

Carl Manaster
  • 39,912
  • 17
  • 102
  • 155
27k1
  • 2,212
  • 1
  • 22
  • 22