Feasibility Study for Potential fP Software - Part #3 (final for the day)

[Fairlight]

Is it just me, or did Jay Ashworth say:
> On Sat, Sep 16, 2006 at 12:37:40PM -0500, Keith' Weatherhead wrote:
> > In thinking as much as I would like to say implement on anything 
> > from 5.0 onward, I do not think you would want to go back to 4.8 and 
> > I am thinking that you might want to restrict it 5.6 forward, 
> > especially if you can get the missing features (@delete  and @time 
> > w/ additional granularity).

Yes, because they've been -so- good at listening to requests over the years
that an officially bug-reported default mode 0666 on EXPORT files from
-three- years ago has yet to be resolved.  If they can't change two bytes
in a mode when opening a file, what again are the odds that I'm going to
get @delete implemented when -I- don't even buy their software?  :)  My
clients do, I don't.  Ergo, I don't technically exist to them.

> > Additionally, I would think that you are going to want to require at 
> > least one if not two dedicated indexes that could be hidden for the 
> > transaction system.  Now that there are 26 indexes with 5.6 you 
> > could require say index Y and Z be devoted to the transaction system.

The idea is to be as non-intrusive as possible, really.  I've got so many
variations on models in my head right now, it's not funny.  And more since
reading your posts.

Can I assume by time delta, you're simply talking timezone offsets from GMT
so that you can run a system in EDT and one in PDT and adjust the data so
that it's the same at the time level?  Or are you talking time delta as far
detailed (in the transaction log you're mentioning) as what NTP would give
you for an offset?  I'm thinking the former, as I see no need for anything
as accurate as the latter, given the fact filePro itself doesn't track
things that accurately.  Just making sure...

Interesting point, though.  I hadn't thought about maintaining the
system-maintained fields like @CD, et al.  Don't ask why, but I wasn't
thinking about the headers; I was thinking about purely the user data.  But
I agree (and that was really helpful advice already in this discussion)
that one would need to preserve all the system maintained fields one could.
That said, those aren't actually settable from inside fP processing, are
they?  You can't actually -change- @CD, for example, can you?  If not,
we're down to rewriting key and data segments--and worse, indexes--from the
ground up, which is a flat-out non-starter.  The idea was to do most of
the work inside fP itself.  If the fP-API that never materialised (after
what, two years now?) isn't there to be used, I can -not- see writing what
amounts to a clone of fP's internal engine just to do this.  Then I'm going
to agree with Jay and say it's really not the way to go about it.

> What *I* am thinking is that it seems like it would be a helluva lot
> easier, not to mentio more productive -- on the assumption that someone
> has a paying customer for it -- to just wave money in fpTech's face,
> and get ODBC-client wired into the Unix runtime tools.

Except ODBC sucks? :) I've used UnixODBC...when I developed FairPay for
PayPal IPN processing and it was the only way to talk to MSSQL.  It's slow
as molasses, Jay.  I mean -really- slow.  It was explained to me that
ODBC is really like two abstraction layers away from what actually needs
to be done, which adds a lot of overhead and drags performance.  That,
and the ODBC cursor is -REALLY- evil.  It certainly had a problem with
multiple query handles concurrently when I used it.  That said, you're
likely correct about reusing standards.  Of course it took them five years
to do the Win32 version of fPODBC, and they used someone else's library to
boot--which, I gather, is why they can't just roll it into *nix.  Sorry,
but I'm not banking on the company that added a spell checker to a database
as a major release feature to add clustering anytime soon.  It's not a
slam, IMHO, it's realism.  It's opinion based on their past actions, which
speak volumes.  I could be wrong, but I don't think I am.

> Then you can leverage Other People's Code to get the ACID right, and
> you don't have to wake up at 3am to calls from lawyers.

You had to mention lawyers, didn't you?  :)  That right there always puts
me off a bit.  :(

> Efficiency is doing the job right.
> 
> Effectiveness is doing the right job.

No argument.  The question is whether fP is actually doing the job right
currently.  Some of the stuff Keith suggested goes beyond clustering fP
in its current state of handling transactions, all the way to ACID.  I
don't think you can hit ACID even under fPODBC, can you?  Does it support
multiple changes on the remote end and a final transaction commit?  I
mean, in a way it makes sense that it should, but since it doesn't even
internally support it (you can't do a series of partial lookups and then
one MASTERWRITE, or the like), it doesn't make a lot of sense to expect
they supported something just for ODBC that they don't regularly handle.

The salient point is whether such a project should actually stick with fP's
current precision, or try to expand upon it.  In a way, I agree
transactions should be added--but that's something best done inside the
engine, not tacked in...and we all know that.  Now, can we deliver what it
already does?  Possibly.  With a lot of work.  Although when we start
talking about maintaining system maintained fields as well, I'm suddenly a
lot less confident about many of the models I was considering.  And a lot
of the external models, even partial-external models suddenly look like
they'd take a huge performance hit unless one rewrote a clone of fP's data
storage and retrieval layer.

Let's just assume there's a way around that.  The question is still whether
or not fP should be extended, or just coupled.  Is it done "right"?  You'll
get two answers to that; yes, for the job it's designed to do--and no, it
doesn't meet with higher standards set by newer software.  And Keith's
answer touched on a lot of technical points that seem to indicate extension
is desirable, "...while one is at it," as it were.  I both agree and
disagree.  I'll suspend firm opinion until I read more.

> I know this isn't the category of answer you were looking for, Mark,
> but you knew I was going to give it to you anyway, right?  ;-)

Actually, I went into this discussion with no expectations other than the
likelihood that Smitty will give me a hard time somewhere along the way.  :)
I'm keeping an open mind right now.

I'm trying to condense the thread a bit, here.  I'm curious why the mixed
financial viability rating from Keith.  On one hand it wouldn't be worth
much, and on the other it would eventually be worth a lot?  That seemed a
bit contradictory.  Unless it's based on initial vs mature.  As I see it,
you can't really screw around with something like this--it's going to have
to be right as possible straight off or it's not worth having.

Let's be frank, here...  -IF- I could pull off such a venture, we're
talking months of development and testing.  We're also talking about
something that vastly impacts scalability.  I was thinking mid 4-digits as
a minimum.  Say $5k.  That's more than a lot of fP installations alone, and
you -still- have to buy each copy of filePro.  But it's only numbers like
that which would really make it viable, as the volume isn't there, period.
It's a small niche within a small niche.  I'm not saying it's a hard and
fast bare minimum, but it's about what came to mind for any kind of
enterprise based solution in -my- mind, given the kind of functionality it
could bring.  If numbers in that range aren't viable, I'm not sure it's
worth doing for general consumption.  This needs to be about realism on
both sides; is it worth doing in and of itself?  Yes.  Is it financially
viable as a time/cost investment for return?  Unless the numbers are fairly
high, no.  I think you could -easily- eat $10k in custom development costs
before coming even close, and that's -not- at top-of-the-food-chain
programmer rates.  It's obviously not a trivial project.  I wouldn't expect
the compensation to be trivial--especially without high volume, in a
specialised marketplace.  Now, is my thinking way off-base, or am I
thinking reasonably?

Which is rather like putting the cart before the horse--ideally one has a
product first before one decides pricing.  Of course, the flip side is that
you don't invest thousands in R&D without a guaranteed return that's far
higher.  So while it feels very much like talking about vaporware (at this
stage it -is- all hypothetical), it's necessary to think that way to see if
it's even worth planning out fully, much less starting in earnest.

> 	"That's women for you; you divorce them, and 10 years later,
> 	  they stop having sex with you."  -- Jennifer Crusie; _Fast_Women_

I find it's a bigger problem when you break up with them and then 2 months
later, they start.  :)

mark->