Packages for APL64

This topic is specifically for discussions on the new APL64 Project currently in development. This topic is open for all to browse. However, to post, one must have a registered account on the APLDN forum only available to APL+Win licensee under a current APL+Win Subscription.

Moderators: Tech Support, phpbb_admin

Packages for APL64

Postby Eric.Lescasse » October 12th, 2018, 12:17 pm

I am wondering if it would be feasible to create the notion of Packages in APL64.

A lot of the success of R and Python comes from the huge amount of already done Libraries or Packages that augment the language and allow a developer to almost do anything he can think of with little efforts.
For example, in R, anyone can load and install any given package among hundreds using the simple: install.packages command and then load the package into memory to use it with the: library command

A lot of us have written nice APL utilities, and there are tons of them out there (developed by Brent Hildebrand, Davin Church, Rex Swain and many others), but they are not centralized in one unique repository, maybe too hard to find for a common APL user, and sometimes they are not documented or not enough documented to be easily usable by anyone.

Most importantly, there is no mechanism built into the language to easily find, access and load them.

What I would envision is something like:

    an easily accessible centralized repository (as simple as a User Command File in the DropBox would be)
    rules for documenting utilities so that they all would be documented using a similar way
    a simple submission mechanism for publishing a utility
    maybe a moderator that would control submissions

What do you all think?

PS
Personally I would be more than willing to give out some of my utilities (which I already started to do with LC.Charts and LC.zREngine)
Eric.Lescasse
 
Posts: 52
Joined: February 8th, 2007, 7:55 pm
Location: Paris, France

Re: Packages for APL64

Postby Davin Church » October 12th, 2018, 1:23 pm

A fine concept, but I can't envision how it would be implemented. Many people have tried this before and nothing has come of it. Rex Swain even tried a few years back and he and I couldn't even come to a consensus on how best to represent dates, much less the utilities needed to process them.

Another issue is that APL programmers are SO very different from one another that even the style of utilities are incompatible with one another. Also, in most other languages it's so difficult to build such libraries that people will go out of their way to use something that's already built, while APL makes it easy to "roll your own" and almost every APL programmer has done so already. Most people, I think, don't much like the idea of using other people's stuff when they already have and use most of what they need and would rather build their own for anything they don't have. So there's a big mindset problem out there that we need to contend with as well.

OTOH, I don't think that building it into the language is necessary. An APL-programmed interface to whatever we want should be easy enough to implement and use, such as with a user-command file.
Davin Church
 
Posts: 543
Joined: February 24th, 2007, 1:46 am

Re: Packages for APL64

Postby Ajay Askoolum » October 14th, 2018, 4:02 am

APL already has packages in the guise of user commands.

I would like to see data structures, like R's Data Frame as a standard means for

1. holding data ... named elements
2. passing it as a parameter to functions
3. printing
4. transfer.

Nested arrays, for all its obvious merits:

1. has too many pathways for construction.
2. is too anonymous -- its elements impart neither a name nor a purpose

Visual Basic, VBScript, VBA, C# etc all have nested arrays but are not as critical to their context as with APL, precisely for the reasons above.
Ajay Askoolum
 
Posts: 743
Joined: February 22nd, 2007, 2:16 am
Location: United Kingdom

Re: Packages for APL64

Postby Davin Church » October 14th, 2018, 12:20 pm

I think being able to add names (and/or attributes) to elements of a (probably nested) array would have it's advantages, but I'm not sure how to make it work without a radical redesign in APL functionality. C-type structures seem pretty artificial to me in terms of APL's basic design. Perhaps something could be worked out by making it part of the .Net interface extensions?
Davin Church
 
Posts: 543
Joined: February 24th, 2007, 1:46 am

Re: Packages for APL64

Postby Marc-Andre.Gaveau » November 3rd, 2018, 7:13 pm

I agree with Eric Lescasse about the feasibility to create the notion of Packages in APL64.
I have been using APl for about 40 years, as a scientific researcher and not a developer. So my concerns can be a little bit different.
Most of students or young researchers learn Python for several reasons : 1) it is free, 2) there are a lot of packages allowing for example to acquire data, to process them and visualize the result.
When I used APL as a PhD student, I found the language very convenient to process the experimental data :thumbsup: . But It was more difficult for the visualization (firstly I used Rainpro) and about acquiring data .... :roll:
Marc-Andre.Gaveau
 
Posts: 1
Joined: March 15th, 2007, 10:02 am
Location: Saclay - France

Re: Packages for APL64

Postby Ajay Askoolum » November 3rd, 2018, 7:50 pm

Right at the outset of the APL64 thread, I suggested that APL should have a data structure that is simple to understand. Nested arrays are fine but they hold data in an unstructured way and anonymously i.e. the data elements do not have names.

I first came across data frames when I looked at R; with the appropriate packages loaded, it is easy to do very powerful things using data frames with very little effort.

More recently, I've started to look at Apache Spark in anticipation of SQL Server 2019; Python (Panda) data frames figures prominently in Spark tutorials.

As I understand it, packages are simply compiled code. In the case of R, it is complied C code. On being loaded, the packages extend the scope of the host environment. I see the same concept working with Notepad++.

I am not at all sure how it might work with APL. The closest I can get to adding extensions with APL is by using []cse which can add a reference to a DLL and the functionality within the DLL becomes accessible within []CSE code. COM provides another way of extending the reach of APL.

Are you expecting/asking for a facility for loading R or panda packages into APL64? What APL objects will such packages act upon?
Ajay Askoolum
 
Posts: 743
Joined: February 22nd, 2007, 2:16 am
Location: United Kingdom

Re: Packages for APL64

Postby Davin Church » November 3rd, 2018, 10:30 pm

Why not just build .NET assemblies and connect them to the APL workspace with ⎕NA? With some of the proposed APL64 extensions, such assemblies could well be written in APL themselves. (I also made a suggestion earlier that custom-built modules like that might be linked directly into the interpreter, and be used much like a ⎕-function is used today.)

If you have assemblies (or other forms of packages, including ordinary APL code) that you want people to use, and you want them distributed widely and automatically, then what you need is a repository application that anyone can use to find and retrieve such packages, much like the Linux world has. You'll also need someone to be responsible for hosting such packages (which may be the more difficult proposition). The application itself could be a simple APL program, optionally made available as a user-command or something similar. I know the code for such an application could be done relatively simply because I already have one. This is what Eric was apparently talking about in his original post, but I see no need to build it into language but just provide the application wherever it's desired.
Davin Church
 
Posts: 543
Joined: February 24th, 2007, 1:46 am

Re: Packages for APL64

Postby Ajay Askoolum » November 4th, 2018, 4:18 am

Why not just build .NET assemblies and connect them to the APL workspace
To do so means reverse engineering the workings of packages (in Python, R etc) in isolation of the life-cycle of the same packages in their native environment. You will always be lagging behind and sooner or later, you will introduce inconsistencies.

This does not enhance APL's rapid application development credentials and begs the question - why not just use Python or R?

I suppose the real question is whether APL64 can use the rich functionality so readily available in data science today? I am aware of the possibilities with R; the options would be to 1. either use R as a COM server 2. write .NET DLLs that wraps R functionality into methods and call them for APL using []CSE.

If APL is going to grow its own packages (i.e. on the basis of user contributions) -- for packages read algorithms/visualization -- there needs to be a common reference framework in terms of the data that such packages will manipulate. Like R, Python etc, APL needs it own data frame (which could be copies of the R or Pythin varieties) supported within the interpreter.

Marc-Andre also gave the thumbs down (or rolling eyes, suggesting bewilderment) to APL's ETL capabilities. E(Extract) T(Transform) and L(Load) is a set of tools that facilities data acquisition for external sources e.g. CSV, JSON, XML etc. as file sources and databases as relational sources.

APL's credentials with arrays with APL64's bigger workspaces makes it an ideal candidate for working with the hybrid sources of data in what has come to be known as the data lake. How will APL64 bring the data into the workspace?

The APL64 Q&A document suggests that the answer is in []wi which can access either third-party COM objects or []CSE for custom tools. Personally, I do not think that this is adequate. Why? Because it requires an impractical leap in skill sets (for a community accustomed to a quad function for everything, which is not the answer) without intuitive worked examples. Imagine functions like fromCSV, fromXML, fromJSON, fromDB, fromNoSQL that read data sources and create an APL data structure that is consistent--and ready for APL's array functionality--irrespective of the source read. Consistency needs to apply across workspaces and across users; then we have a common framework of reference for building tools or packages that act on the unified data structure.
Ajay Askoolum
 
Posts: 743
Joined: February 22nd, 2007, 2:16 am
Location: United Kingdom


Return to APL64 Project

Who is online

Users browsing this forum: No registered users and 1 guest

cron