Timeout enhancement requests

[Davin Church]

I know that my previously reported timeout problems are still being looked into, but I'd like to make a request for an enhancement that may be able to be done at the same time as any bug fixes.

Right now, if a one-to-one (or hung) session times out, the servicing interpreter stays active in memory and the workspace is simply reloaded. This works fine for simple situations where all the user state information is kept in variables in the workspace, but does not work so well for more persistent information in the environment outside the workspace. For instance, native and APL file ties survive workspace loads and continue to be tied when the workspace is reloaded. GUI objects such as timers and printers and external ActiveX controls also survive to the new session. Database logins, whether they're ActiveX-based or not, are another example category of persistent data.

Granted, some things (like file ties) can be removed upon reloading (providing that the application is not overly complex), but other things can't or are much more difficult to remove. For example, some older kinds of database connections need to be actively commanded to shut down and can't be done automatically.

Therefore, when resetting a timed-out session, the entire interpreter should be deleted and restarted rather than just relying on a workspace )LOAD to be sufficient. Shutting down the interpreter will release all resources that CAN be automatically released, and will improve the situation considerably. (The new interpreter can actually be started first to save time getting it up again.)

In addition, actually shutting down the interpreter should provide another major benefit as a side effect. It should (I hope) cause the APL system object's ('#') onDestroy event to fire when it begins shutting down. This will give the active workspace notification that it's about to go away and allow it to shut itself down cleanly, potentially even releasing or cleaning up special resources that cannot be released automatically. It can also be used to perform end-of-session logging that can't be accomplished in any other way. (If the onDestroy event doesn't fire automatically, please see that it or something comparable is specifically invoked anyway.)

Do you think that this enhancement might be implemented sometime in the near future? I wouldn't expect it to be very difficult to do - perhaps as little as a few lines of code. Of course, I could use it right away myself but I would think that anyone using the one-to-one feature could benefit immediately from this new capability.

[joe_blaze]

Davin: "I know that my previously reported timeout problems are still being looked into..."

Joe: Timeouts are related to the detection and interpretation of information to designate a thread as 'non-active'. This detection and interpretation phase can be affected by events which are external to the web server, so that timeout settings are approximate and can vary a bit. As far as it was possible to duplicate your situation, no extreme variation in timeouts was observed by us.

==================

Davin: "Right now, if a one-to-one (or hung) session times out, the servicing interpreter stays active in memory and the workspace is simply reloaded."

Joe: The APL+Win workspace is not always 'simply reloaded'. When APL WebServices receives and interprets the information that causes it to designate a processing thread as 'non-active', APL WebServices makes the decision to end or persist the .Net processing thread which commenced the associated subordinate APL+Win ActiveX session based on the programmer's configuration of the MinPool and MaxPool settings.

For example, suppose APL WebServices detects a 'non-active' thread:

(a) If the number of existing threads, including the 'non-active' thread is greater than the MaxPool setting, the 'non-active' thread is ended.

(b) If the number of existing threads, including the 'non-active' thread is less than or equal to the the MinPool setting, the 'non-active' thread is 'persisted' and returned to a state in which it can accept incoming client requests. APL WebServices makes a request to the associated APL+Win ActiveX session to load the programmer-associated APL+Win workspace.

The implementation of MinPool, MaxPool and the above-described behavior of APL WebServices is designed to maximize performance, by minimizing unnecessary loading of the interpreter or workspaces. The enhancement you are requesting would significantly diminish the throughput of a web services application because the interpreter would be loaded many more times than necessary. The definition of 'necessary' is design-based and assumes that the server-side programmer has planned for unanticipated errors.

Keeping the MinPool setting low with respect to client user load will generally cause the interpreter to be reloaded which is evidently the behavior you are looking for, although such a setting is likely to degrade performance of the web service and make the client experience less appealing.

The 1:1 option for a client to access a specific server-side thread repeatedly, is still subject to the same timeout behavior of non-1:1 situations. Of course if the APL WebServices configuration 'Debug' setting is '1', then timeouts controlled by APL WebServices are not considered, but generally Debug = "1" would not be a production environment setting.

I don't know what you mean by 'hung' session - errors in the server-side program?

==================

Davin: "This works fine for simple situations..."

Joe:

To facilitate efficient use of server-side resources, APL WebServices implements asychronous multi-threading. On the Windows platform this means using .Net, so APL WebServices is a .Net program. Microsoft provides the ActiveX 'interop' way for .Net programs, like APL WebServices, to access Win32 programs, like APL+Win. Conveniently, APL+Win has a robust ActiveX interface which can be used by APL WebServices.

A web server is effectively a scheduler. Once the incoming client request is scheduled, i.e. associated with an active server thread, the server-side
program, e.g. in the APL+Win worksapce, is on its own to work 'properly' or fail within the allotted timeout period. When that thread's time is up,
either as a result of a programmer-established timeout, an external timeout or error, a client abandonment of the session or an error in the server-side application program, the web server ends that thread or persists it according to the behavior described above based on the MinPool and MaxPool settings.

The ending or persisting is done by the web server without regard to what the subordinate APL+Win server-side application program is, or should have been, doing.

Timeouts and errors are only one example of unanticipated errors. For example, a thread can become 'non-active' when the client closes the browser or the ISP goes into maintenance mode. It is fundamental to a client/server application that the server-side programmer designs the application program to handle unanticipated errors. It is necessary for the server-side programmer to assume that at any time the the client, the server or both are going to fail without warning.

For example, when data-base access is used on the server-side, a database which supports 'transactions' which are initially journaized and placed into a temporary database location and rolled back unless finalized is a necessity. Some of the older and possibly some royalty-free database access technology does not provide for 'transactions' and so is unsuitable for server-side programming.

The structure of the client-side GUI must often be crafted in 'wizard-style', so that there are well-defined, recoverable steps for the user with mileposts which can be identified by the server-side in subsequent client sessions.

When a 'non-active' thread is ended, there is no possible warning to the APL+Win workspace. This means that the Close or Destroy events are not going to fire in the APL+Win workspace. When the .Net thread is ended, what the thread was, or was supposed to be, doing is not known to APL WebServices and therefore cannot be considered. APL WebServices does not [and should not] 'request' that APL+Win close using an APL+Win ActiveX method, because APL+Win could refuse or delay the request and thereby tie up server-side resources out of the control of the web server.

The requirement that the server-side application programmer prepare for unanticipated errors is not peculiar to APL WebServices, but is part of the web services style of programming in which the server-side application program is subordinate to the web server software.

[Davin Church]

Davin: "I know that my previously reported timeout problems are still being looked into..."

Joe: Timeouts are related to the detection and interpretation of information to designate a thread as 'non-active'. This detection and interpretation phase can be affected by events which are external to the web server, so that timeout settings are approximate and can vary a bit. As far as it was possible to duplicate your situation, no extreme variation in timeouts was observed by us.

I believe I reported that:

In my sample log, you'll notice that my first session goes for over 10 minutes of idle time (the limit is set to 180 seconds) without reloading. Then after restarting the server, it's reloading after only 39-48 seconds of idle time (in these tests).

This does not sound to me like this is an issue with a slight variance that can be caused by minor interference from events external to the web server. (And since this is the only thing running on the web server with 8 CPUs, I can't see how anything short of a massive workload could change anything more than a few milliseconds.)

My symptoms, as demonstrated to you in my log files, are considerably more serious than your explanation seems to be able to account for. Therefore, I think something else is going on that we need to find.

Is there any other testing I can do for you to help the process along?

[Davin Church]

Davin: "Right now, if a one-to-one (or hung) session times out, the servicing interpreter stays active in memory and the workspace is simply reloaded."

Joe: The APL+Win workspace is not always 'simply reloaded'. When APL WebServices receives and interprets the information that causes it to designate a processing thread as 'non-active', APL WebServices makes the decision to end or persist the .Net processing thread which commenced the associated subordinate APL+Win ActiveX session based on the programmer's configuration of the MinPool and MaxPool settings.

Well, that's an interesting statement, since it isn't supposed to EVER have more than MaxPool sessions active, right? So how can it be in the situation where it gets closed down instead of reloading?

But assuming I changed my preliminary statement to read "...if a session times out and there are not more than MaxPool sessions active...", I don't see how that affects my enhancement request.

[Davin Church]

For example, suppose APL WebServices detects a 'non-active' thread:

Just for processing-logic consideration, you haven't said what happens if the existing thread count is between MinPool and MaxPool.

[Davin Church]

The implementation of MinPool, MaxPool and the above-described behavior of APL WebServices is designed to maximize performance, by minimizing unnecessary loading of the interpreter or workspaces. The enhancement you are requesting would significantly diminish the throughput of a web services application because the interpreter would be loaded many more times than necessary. The definition of 'necessary' is design-based and assumes that the server-side programmer has planned for unanticipated errors.

This is quite true, if you're assuming that the sessions are openly assigned and are NOT one-to-one sessions. And this behavior seems perfectly reasonable to me and quite efficient for such "openly available" threads. However, the situation changes significantly when you're ending a one-to-one thread. We must assume that state information is being kept in a one-to-one session and that leads to the assumption that such state information really needs to be cleaned up as thoroughly as possible when it is forcefully ended outside of application control. The only way to perform this efficient of a cleanup is to end the thread rather than reusing it (which can accidentally and erroneously preserve some of this state information).

Depending on the situation, the server-side programmer CANNOT ALWAYS plan for and recover unanticipated errors, therefore it should be the responsibility of the environment to do it for him and/or provide him with enough signals so he can always do it himself.

Therefore:
(1) My request will not affect the behavior or speed of servicing non-1:1 sessions at all.
(2) My request would seem to be necessary if you are to allow complex 1:1 applications to be written for APL Web Services.
(3) My request should not impact the speed of servicing 1:1 sessions overall very much because this situation will rarely happen (compared with the overall load of the web site). [Detailed reasoning upon request.]
(4) If you implement my trivial MinPool change request I made elsewhere, then it will not affect the performance of servicing 1:1 sessions at all.

Therefore, I don't see any technical reason why this couldn't be done, unless you can point out something that I've missed. (Business reasons are another matter, but I'd think you'd want to be able to support these kinds of applications.)

[Davin Church]

Keeping the MinPool setting low with respect to client user load will generally cause the interpreter to be reloaded which is evidently the behavior you are looking for, although such a setting is likely to degrade performance of the web service and make the client experience less appealing.

No, this is not the behavior I'm looking for at all. I want plenty of unassigned threads to stay available, especially for single web requests. I'm only asking for a change in behavior when forcefully ending a 1:1 session, which should happen very rarely and affect performance virtually not at all.

[Davin Church]

The 1:1 option for a client to access a specific server-side thread repeatedly, is still subject to the same timeout behavior of non-1:1 situations.

As I understand it, there are two reasons to "time out" a thread (and please tell me if my nomenclature here is poor):

(1) The thread (either kind) doesn't return a result to AWS in a reasonable time so its considered to have "hung" in the application and therefore has to be forcefully removed.

(2) A 1:1 thread has not received any web requests in some lengthy period of time, so it is considered to no longer be in use and is discontinued (either shut down or restarted by reloading).

We seem to be having a communication issue here, and your statement above doesn't agree with my understanding of the situation, so let me start over by asking a direct, simple question:

Is this much correct?

If so, all my discussions are really about situation (2) above. I'm trying to avoid discussion of situation (1) as much as possible.

[Davin Church]

I don't know what you mean by 'hung' session - errors in the server-side program?

I simply meant that the server-side application thread is not returning a result from a call in a timely manner, presumably due to errors in the APL code. You seemed to be emphasizing that situation in some of your replies so I wanted to make sure I included it in my reply - that's all.

[Davin Church]

Davin: "This works fine for simple situations..."

Joe:

To facilitate efficient use of server-side resources, APL WebServices implements asychronous multi-threading. On the Windows platform this means using .Net, so APL WebServices is a .Net program. Microsoft provides the ActiveX 'interop' way for .Net programs, like APL WebServices, to access Win32 programs, like APL+Win. Conveniently, APL+Win has a robust ActiveX interface which can be used by APL WebServices.

A web server is effectively a scheduler. Once the incoming client request is scheduled, i.e. associated with an active server thread, the server-side
program, e.g. in the APL+Win worksapce, is on its own to work 'properly' or fail within the allotted timeout period. When that thread's time is up,
either as a result of a programmer-established timeout, an external timeout or error, a client abandonment of the session or an error in the server-side application program, the web server ends that thread or persists it according to the behavior described above based on the MinPool and MaxPool settings.

Yes, that is all understood.

The ending or persisting is done by the web server without regard to what the subordinate APL+Win server-side application program is, or should have been, doing.

I agree with that, too, with one exception... 1:1 sessions (which AWS knows about) need to be treated slightly differently than instantaneous sessions, as mentioned in a previous message.

Timeouts and errors are only one example of unanticipated errors. For example, a thread can become 'non-active' when the client closes the browser or the ISP goes into maintenance mode. It is fundamental to a client/server application that the server-side programmer designs the application program to handle unanticipated errors. It is necessary for the server-side programmer to assume that at any time the the client, the server or both are going to fail without warning.

Agreed, except in some complex situations it is NOT POSSIBLE, even in theory, for the server-side program to handle ALL unanticipated errors without SOME kind of notification. Therefore, if APL Web Services is expected to be used for these kinds of applications, some method must be made available to provide such notification to the application (when possible) or AWS should kill the task as a last resort to force a clean restart.

Operating systems have been using this method for decades to end misbehaving tasks - I don't see why it won't work here.

For example, when data-base access is used on the server-side, a database which supports 'transactions' which are initially journaized and placed into a temporary database location and rolled back unless finalized is a necessity. Some of the older and possibly some royalty-free database access technology does not provide for 'transactions' and so is unsuitable for server-side programming.

Exactly, but I have to be able to know when to roll back those transactions. If you just reload the workspace without telling me you're going to do it, I'm still inside a transaction and don't know it. How am I supposed to know when to roll back the transaction?

The structure of the client-side GUI must often be crafted in 'wizard-style', so that there are well-defined, recoverable steps for the user with mileposts which can be identified by the server-side in subsequent client sessions.

No problem - that's the whole point of having 1:1 sessions.

When a 'non-active' thread is ended, there is no possible warning to the APL+Win workspace. This means that the Close or Destroy events are not going to fire in the APL+Win workspace. When the .Net thread is ended, what the thread was, or was supposed to be, doing is not known to APL WebServices and therefore cannot be considered. APL WebServices does not [and should not] 'request' that APL+Win close using an APL+Win ActiveX method, because APL+Win could refuse or delay the request and thereby tie up server-side resources out of the control of the web server.

Here I have a disagreement. It sounds perfectly possible and feasible to me for APLWS to send a Close/Destroy/Delete request to the application first (Close is not necessarily a requirement, and the returned result would be ignored anyway). If the application does not respond to the request within several seconds (which would only happen if it were "hung"), THEN it can forcefully kill the thread. But notification would have been made and given the application at least an opportunity to shut itself down cleanly.

The requirement that the server-side application programmer prepare for unanticipated errors is not peculiar to APL WebServices, but is part of the web services style of programming in which the server-side application program is subordinate to the web server software.

Yes, but no other server-side application systems that I know of have the option of )LOADing a workspace in an existing thread to recover from a failure. APL Web Services is unique in this regard. If APL Web Services recovered from a catastrophic failure in the same way as all other server-side systems do, by killing the thread, this would be sufficient for APL Web Services (and me), too. I would prefer prior notification if feasible, but killing the thread outright would at least accomplish the goal. Simply reloading the workspace to recover from a drastic failure is NOT an acceptable method, and is (because of unavailability) never done anywhere else on the web.

Drastic failures should not happen often, and killing the task therefore should not impact overall client-server performance. If a 1:1 session is ended normally (by returning a 0 flag), then reloading the workspace is a perfectly reasonable and fast recovery option, because the application would have already shut itself down cleanly.

I still don't understand why you think this recovery method is a problem.

[joe_blaze]

When is an APLWebServices processing thread not active?

Currently the primary determinant used by APLWebServices to mark a processing thread as inactive is the expiration of the programmer-set timeout. Note that the web server's strategy to not degrade performance means that it may defer checking for a timeout in favor of handling a new incoming request, so that ending a thread based on a timeout may not occur immediately upon the expiration of the programmer-set timeout.

Can/Should APLWebServices send an 'I am about to kill you' message to the APL+Win ActiveX session underlying a non-active processing thread and should it wait for a response from APL+Win?

Currently there are no messages originating from APLWebServices which are sent to the processing thread. All messages sent to the server-side processing thread by APLWebServices originate on the client side. This is consistent with mainstream web server software like Apache and IIS.

If APLWebServices were to send the message 'I am about to kill you' to the underlying APL+Win ActiveX session, that suggests that the underlying APL+Win ActiveX session should have some control over APLWebServices which started the APL+Win ActiveX processing thread. Modifying the message to 'I am about to kill you in #### milli-seconds' just adds the server-side processing performance burden to check when this postponement of a timeout has expired.

Note that there is no monitoring by APL WebServices of the processing being done by the APL+Win ActiveX session underlying a particular processing thread. APLWebServices starts and ends processing threads based on the programer-set configuration. By design APLWebServices is agnostic with respect to the 'work' going on under a processing thread started by APLWebServices. This design was selected, following Microsoft IIS designs, so that some versions of APLWebServices can start other server-side applications besides APL+Win, such as .asp or .aspx processes.

From the server-side application system viewpoint the server-side programmer should establish processing times for the business rules milestones of the application and set the timeouts appropriately. It is important to establish processing performance standards for any web service as many surveys have indicated that client attention spans for browser-based software do not extend beyond a few seconds. Suggesting that APLWebServices killing the processing thread without notifying APL+Win is a problem in this scenario, really means that the programmer-set timeouts were not appropriate or the server-side application system processing needs performance tuning to stay within reasonable limits. It is assumed that the server-side application processing granules are identified and exposed to the client as discrete, recoverable, processing steps each with appropriate processing performance standards and their associated timeouts.

Another scenario of concern from the server-side application system viewpoint would be when communication activity is stopped between the client and server, e.g. due to the client abandoning the session, etc. In this case it is not possible for APLWebServices or any entity to notify the server-side application that this has occurred prior to the expiration of the programmer-set timeout. The web server cannot distinguish this ‘crash’ condition from a timeout which occurred because the server-side process took too long.

Server-side applications often incorporate scheduled batch jobs to eliminate orphan database records, partially-complete transactions, reversal of incomplete intermediate processing milestones, etc.