[nzlug] Data processing.....

[Cliff Pratt]

Andras Farago wrote:
> Hi guys, What software can you recommend to process a database/txt
> file/whatever with approx 20 million rows, 25 fields in each row? I
> need to perform only very simple queries like “SELECT a, Sum(b) FROM
> data GROUP BY a ORDER BY Sum(b) DESC;”. Another important option is
> the time, I can’t wait all day to get the result.
> 
If you loaded it into a database you could build an index on it and get
quick results that way. Though loading it and building the index could
take some time.

Cheers,

Cliff