User command in @key processing

Brian White bw.aljex at gmail.com
Mon Aug 13 18:22:27 PDT 2018 

It's useful that it doesn't close unless you tell it to, because it's
useful to set up a co-process that gets re-used many times rather than
forking and closing every time.

user()is a very tricky feature to use *reliably* though. You must always be
absolutely sure about the sequence of read vs writing to to the child
process. It's stupid easy to get out of sync where you are trying to write
to the processes stdin when it's not reading, or read from the processes
stdout when it's not writing, and you sit there hung forever. All it takes
if for the child program to issue one line of output that you didn't expect
(like an error message, or a response that normally has x lines but under
some odd condition that doesn't usually happen can be some other number of
lines), or fail to issue a line of output you did expect. It's very useful,
but it's also very fragile.

It's possible to set things up so that it's reliable, but you have to
actually set things up and do things right yourself.

For myself, the way I do it is I follow these rules:
For the child program, I make a shell script which loops until directed to
exit, or until the parent exits, and the shell script absorbs all potential
irregularities from any programs run inside the script. Then the script
itself is the only thing that reads from stdin or writes to stdout, and I
make sure the loop in the script always reads and writes in exctly the same
pattern no matter what any of the programs in the script did.

For instance, if you issue say a command and it didn't return anything at
all, that would hang fp, because in fp you'd have to try to read at least
one line, but if the command didn't send at least one linefeed, then fp
would sit and read forever.

But in a shell script you can x=`command`
and then echo $x, and the echo command will append it's own linefeed
regardless if $x is even unset (because, say that line in the shell script
never even got run because of some bug), or is set but totally null, or
contains a string with no linefeed, or contains a string with multiple
linefeeds but doesn't end with a linefeed, or is a normal expected form of
value.

Basically use the shell to normalize the input and output between filepro.

Then, in filepro, I stick to a certain boilerplate example of hour to ever
user user(), in all cases.

After setting up the looping shell script, I write a simple gosub with no
conditional spaghetti within the gosub itself, so the gosub always does
exactly the same sequence of events with no risk of getting it wrong under
some unexpected condition. And the gosub goes like, essentially:

user prog = prog
prog = x
x = prog
return

If the shell script has some other pattern like read one line of input then
write 4 lines of output, then the gosub would be

user prog = prog
prog = x
xa = prog
xb = prog
xc = prog
xd = prog
return

The key points are:
* No If: at all within the gosub if I can possibly avoid it.
* Gosub always includes the user foo = foo line before the reading &
writing within the gosub.
* All reads/writes to the user alias are done within the gosub. None at all
outside of the gosub.
* No multiple exits from the gosub, or indeed any conditional/variable
behavior within the gosub. It always does the same thing from beginning to
end.
* The user() alias is never closed.

Then if I adhere to that template religiousy, I can just use the gosub as
much or as little as I want, and I am never at risk of hanging trying to
read when the child is not writing, hanging trying to write while the child
is not reading, creating a zillion zombie children, spawning the child
process for no reason if I never used it.

I arrived at this template procedure because of a few things:
* If the user() is not closed, then re-running the same user command (same
alias) has no effect, so it's harmless to run 600 times. Placing it within
the gosub at the beginning gurantees that the command has been run before
any attempt to read or write to the alias.
* closing the alias isn't a reliable way to close the child process. When I
used to close the alias each time, it frequently created a zillion zombie
children. fp would sort of close the process but they didn't actually go
away as long as the clerk or report process was still running, then the
next time the user() command was run it would spawn a new child process,
repeat, repeat, repeat... creating a zombie child process each time until
the clerk or report process finally exits, which might be a long time
because a user could just move from record to record inside the same clerk
session all day long, or run a huge report that hits a lot of records
etc... This is probably because of some glitch with waiting output on
stdin, stdout, or stderr that needs to be consumed one way or the other,
or, who knows what.
* You can practically never really count on the output from almost any
commands to really be guaranteed predictable at all times. There is always
some odd situation where it outputs more lines or fewer lines than it
usually does or than you expected, or outputs nothing at all, etc, and
there is no way to test the output from user() for nothingness like you can
in shell. You can read from a user alias and the result could be "", but
that would only be if the program had actually output at least a single
linefeed. It's just like the shell "read" command (it's default behaviour
without using any options) It reads until it sees a linefeed. No linefeed,
then it just waits forever.

It is possible to do it other ways. Like you could have code in the script
that says basically "when I read "EXIT" then, exit", and then have the
filepro code say
foo = "EXIT" ; close foo

That can work too, but I just find it less bullet-proof, not to mention
less efficient in cases where you might call the gosub multiple times
within a clerk/report session.

It is possible to have situations that require the gosub to be more complex
than this example, or even really need your program to do all kinds of
conditional jumping around while interacting with the user() throughout the
whole program, with no gosub, it's just that this is the ideal I always try
to stick to as religiously as possible, and only break this rule if I have
to. I do have one thing I did one time where I made a sort of "driver" to
interface with a network service, where the user command was just "netcat"
talking to a tcp service, and I did all kinds of logic in processing where
I send a command to the tcp service, read the first line of output, and
decide what to do next based on that, and just hope to hell the docs for
that service were complete and accurate! If I had to do that same project
today, I'd probably do more of it in shell and stick to this rule like
usual. I think that program in fact turned out to give that customer some
problems once in a while probably exactly because I didn't. They had their
own in-house developer after that so I never worked on it again but just
explained how it was supposed to work to their guy.

Really, it's not so important exactly how you go about it, never close,
always close, so long as one way or another you do ensure that you are
always in sync with the child process, where you always only ever try to
write when it is definitely waiting to read input, and always only ever try
to read output from the child when it is going to write at least one line
of output, including a terminating linefeed. That's the important part. And
just generally realize that user() requires special care, and isn't really
the best way to run a child process unless you really do wnat to interact
with it to read and write stdin stdout with it. If you just want to run
something and don't really need to collect it's output, use system(). And
even in most cases where you do want to collect it's output, still use
system() and a temp file.

-- 
bkw

On Mon, Aug 13, 2018 at 2:46 PM James Flanagan via Filepro-list <
filepro-list at lists.celestial.com> wrote:

> Mark,
>
> Yeah, that was exactly it.  Thank you very much,
>
>
> James Flanagan
> Flantec.com <http://flantec.com/>
>
> email:  James at Flantec.com <mailto:James at Flantec.com>
> mobile: 760-458-8498
> home:   219-221-6219
> text:           7604588498 at txt.att.net <mailto:7604588498 at txt.att.net>
>
> Skype:  Flantec
> ichat:  James at Flantec.com <mailto:James at Flantec.com>
>
> home:   1901 Cidermill Road
>                 Michigan City, IN 46360
>
> > On Aug 13, 2018, at 2:38 PM, Fairlight via Filepro-list <
> filepro-list at lists.celestial.com> wrote:
> >
> > Your mistake is likely in not using CLOSE.
> >
> > After your "aa=xxx" line, use this line:
> >
> > ::close xxx:
> >
> > Should it -technically- close upon re-hitting the user command?  That's a
> > mattter one could debate either way.  I strongly suspect it is not
> closing,
> > however, leaving you with a dead file descriptor.
> >
> > mark->
> >
> > On Mon, Aug 13, 2018 at 02:02:04PM -0400, James Flanagan via
> Filepro-list thus spoke:
> >> Filepro 5.8.01.06D6
> >> Redhat Exnterprise 6.8
> >>
> >> /bin/test_script
> >> echo 23
> >>
> >> Input processing
> >> ::end:
> >> ::?:
> >> @keyx::?:
> >> ::aa(2,.0):
> >> ::user xxx = /bin/test_script
> >> ::aa=xxx:
> >> ::msgbox aa:
> >> ::end:
> >>
> >> The first iteration of this code, when hitting the ?X? key works and
> the msgbox contains ?23?.  Every subsequent time, however, the msgbox is
> blank.
> >>
> >> I found the following in the manual, but I am not quite certain that
> this is the issue.
> >> IMPORTANT: When reading from a user program, filePro Plus executes the
> program only once, not over and over. Therefore, make sure the user program
> itself loops until it reaches an end-of-file.
> >>
> >> Can input processing keep re-running a user command each time in @key
> processing, however many times the key is pressed for a given record.
> >>
> >> Thank you very much, in advance.
> >>
> >>
> >> James Flanagan
> >> Flantec.com <http://flantec.com/> <http://flantec.com/ <
> http://flantec.com/>>
> >>
> >> email:       James at Flantec.com <mailto:James at Flantec.com> <mailto:
> James at Flantec.com <mailto:James at Flantec.com>>
> >> mobile:      760-458-8498
> >> home:        219-221-6219
> >> text:                7604588498 at txt.att.net <mailto:
> 7604588498 at txt.att.net> <mailto:7604588498 at txt.att.net <mailto:
> 7604588498 at txt.att.net>>
> >>
> >> Skype:       Flantec
> >> ichat:       James at Flantec.com <mailto:James at Flantec.com <mailto:
> James at Flantec.com>>
> >>
> >> home:        1901 Cidermill Road
> >>              Michigan City, IN 46360
> >>
> >> -------------- next part --------------
> >> An HTML attachment was scrubbed...
> >> URL: <
> http://mailman.celestial.com/pipermail/filepro-list/attachments/20180813/98c7c5f6/attachment.html
> <
> http://mailman.celestial.com/pipermail/filepro-list/attachments/20180813/98c7c5f6/attachment.html
> >>
> >> _______________________________________________
> >> Filepro-list mailing list
> >> Filepro-list at lists.celestial.com <mailto:
> Filepro-list at lists.celestial.com>
> >> Subscribe/Unsubscribe/Subscription Changes
> >> http://mailman.celestial.com/mailman/listinfo/filepro-list <
> http://mailman.celestial.com/mailman/listinfo/filepro-list>
> >
> > --
> > Audio panton, cogito singularis.
> > _______________________________________________
> > Filepro-list mailing list
> > Filepro-list at lists.celestial.com <mailto:
> Filepro-list at lists.celestial.com>
> > Subscribe/Unsubscribe/Subscription Changes
> > http://mailman.celestial.com/mailman/listinfo/filepro-list <
> http://mailman.celestial.com/mailman/listinfo/filepro-list>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://mailman.celestial.com/pipermail/filepro-list/attachments/20180813/8925ed56/attachment.html
> >
> _______________________________________________
> Filepro-list mailing list
> Filepro-list at lists.celestial.com
> Subscribe/Unsubscribe/Subscription Changes
> http://mailman.celestial.com/mailman/listinfo/filepro-list
>


-- 
bkw
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.celestial.com/pipermail/filepro-list/attachments/20180813/00027c28/attachment.html>

More information about the Filepro-list mailing list