Filepro-list Digest, Vol 29, Issue 12

[Boaz Bezborodko]

> Date: Tue, 06 Jun 2006 15:11:15 -0500
> From: Steve Bergman <steve at rueb.com>
> Subject: Re: Improving FP performance?
> Cc: filepro-list at lists.celestial.com
> Message-ID: <4485E163.7070301 at rueb.com>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> Hi,
>
> Thanks, Walter, for the reply.  The RAID array wouldn't help a whit in
> this case.  With 2GB of memory, the data all fits in the cache.  You're
> right about my being more familiar with the internals of PostgreSQL.
> But in this case I was thinking of Cobol and C/Isam files and the need
> for a rebuild/export/import from time to time.  IIRC from years back,
> Filepro marks records for deletion but does not immediately remove
> them.  However, if the file is growing, it *can* reclaim that space for
> new records. These files would be growing.
>
> I know that an index is being used to do the initial selection of
> records in the main file.  Is it possible in filepro to do a lookup that
> does *not* use an index?  I don't think there is.  I think you have  to
> specify an index as part of the lookup syntax.  But it *feels* like what
> is happening is a bunch of unnecessary sequential scans.  We're only
> dealing with ~2000 records, here, and not all *that* much processing.
> It should be faster.
>
> To answer your question. I believe that the processing only formats are
> mostly doing record adds and updates.
>
> Thanks,
> Steve
>   
Even though you are working on an indexed selection set you could be 
doing numerous other lookups within the program.  The problem could just 
as easily be a bug in the code that forces the process to go through the 
entire file on a looped, indexed search instead of terminating early 
when the search has gone past the search criteria.  If the lookup is 
nested in another process the time to complete the task would grow 
exponentially with the growth of the data-set. 

For example:  Let's say you need to looking through invoices on the 
basis of the check-number used to pay off the invoice for each check 
number in a set.  If you do a lookup by check number you want to check 
that the check number has gone past the number used in the search.  If 
you don't you'll end up looking through the entire file before getting 
out of the loop and doing the same thing on the next check number.  Cut 
off the search as soon as the check numbers are no longer equal and the 
process is much, much faster.

I know that I've been guilty of such bugs on numerous occasions.

I think you have to look at what the code is doing here before jumping 
to a hardware based solution.

To find this you either have to walk through a debug run of the code or 
place strategically located SHOW statements to show what the program is 
doing when to give clue of where to look.

Boaz