Zip 256-bit AES encryption

[jeff.koloseus]

It would be very useful for the APL+Win Zip class to have the option to use 256-bit AES encryption rather than the much weaker standard zip encryption.

The newer AES encryption is required for any sort of real data security, such as to meet HIPAA requirements. It is available in WinZip, etc., as well as ActiveX products, but it would be a great convenience to have it in APL+Win directly.

I think the needed code is available as open source, as evidenced by this excerpt from the WinZip web site:
"To perform AES encryption and decryption, WinZip uses AES functions written by Dr. Brian Gladman. The source code for these functions is available in C/C++ and Pentium family assembler for anyone to use under an open source BSD or GPL license from the AES project page on Dr. Gladman's web site. The AES Coding Tips page also has some information on the use of these functions. WinZip Computing thanks Dr. Gladman for making his AES functions available to anyone under liberal license terms."

The APL+Win documentation mentions that it uses Info-ZIP. Perhaps there is an updated set of code that has the AES functionality.

Thanks.

[Davin Church]

That sounds like a nice idea. I also use 7-Zip (open-source) for most of my manual zipping needs, but to call it from APL I've been using []CMD (which is a pain). There might be a DLL version of it that can be called more directly or possibly integrated into APL in the same way as Info-Zip.

[Ajay Askoolum]

I've not used 7-Zip hence the question: if you use AES encryption, how many files do you end up with - one or two? In other words, is the encryption key buried inside the zip file or is it held in a separate file?

[Davin Church]

Just one file, the way you'd expect zip-type files to work normally.

[Tech Support]

It would be very useful for the APL+Win Zip class to have the option to use 256-bit AES encryption rather than the much weaker standard zip encryption.

The newer AES encryption is required for any sort of real data security, such as to meet HIPAA requirements. It is available in WinZip, etc., as well as ActiveX products, but it would be a great convenience to have it in APL+Win directly.

I think the needed code is available as open source, as evidenced by this excerpt from the WinZip web site:
"To perform AES encryption and decryption, WinZip uses AES functions written by Dr. Brian Gladman. The source code for these functions is available in C/C++ and Pentium family assembler for anyone to use under an open source BSD or GPL license from the AES project page on Dr. Gladman's web site. The AES Coding Tips page also has some information on the use of these functions. WinZip Computing thanks Dr. Gladman for making his AES functions available to anyone under liberal license terms."

The APL+Win documentation mentions that it uses Info-ZIP. Perhaps there is an updated set of code that has the AES functionality.

Thanks.

It is doubtful that the Zip class will be enhanced until Info-ZIP comes out with an official release (not beta) supporting the 256-bit AES encryption.

[Davin Church]

It is doubtful that the Zip class will be enhanced until Info-ZIP comes out with an official release (not beta) supporting the 256-bit AES encryption.

Any thoughts about providing direct access to 7-Zip via APL as an alternative Zip package?

[Tech Support]

It is doubtful that the Zip class will be enhanced until Info-ZIP comes out with an official release (not beta) supporting the 256-bit AES encryption.

Any thoughts about providing direct access to 7-Zip via APL as an alternative Zip package?

Do you know if 7-Zip provides any supporting interfaces (ActiveX/COM) that could be accessible to APL+Win? If yes, then that's one possible approach that we would recommend to APL programmers to use.

[Davin Church]

Do you know if 7-Zip provides any supporting interfaces (ActiveX/COM) that could be accessible to APL+Win? If yes, then that's one possible approach that we would recommend to APL programmers to use.

I haven't researched it. But since it's open-source it should be possible to compile such a version or perhaps even compile it directly into APL. I wouldn't expect there to be much in the way of licensing requirements (just acknowledgement, I'm guessing). If it were part of APL's zip-system, then it might even be possible to share the interface with an extra property to select which underlying product to use.

Anyhow, it's a thought to consider.

[Ajay Askoolum]

APL+Win with the C# script Engine can create and expand ZIP files out of the box. For demonstration, see the functions below. Incidentally, these functions will need to be enhanced with error trapping and verification of its arguments.

Creating Zip files:

Code: Select all

    ∇ ZipFileName CreateZip ZipDirectory;⎕cself
[1]   ⍝ Ajay Askoolum
[2]   ⎕cself←'c' ⎕cse 'Init' 'System'
[3]   ←⎕cse 'ExecStmt' 'using System;'
[4]   ←⎕cse 'LoadAssembly' 'C:\Windows\Microsoft.NET\Framework\v4.0.30319\System.IO.Compression.FileSystem.dll'
[5]   ⎕cse 'ExecStmt' 'System.IO.Compression.ZipFile.CreateFromDirectory(@{0},@{1});' ZipDirectory  ZipFileName
[6]   ⍝ ⎕cse 'GetLastError' ⍝ Debugging step, if needed
[7]   ⎕cse 'Close'
[8]   →0
[9]   ⍝ verify path to System.IO.Compression.FileSystem.dll; this file is part of .NET Framework
[10]  ⍝ ZipDirectory MUST exist ... check before calling this function
[11]  ⍝ ZipFileName must NOT exist already ... check before calling this function
    ∇


Expanding Zip files:

Code: Select all

    ∇ ZipFileName ExpandZip UnZipDirectory;⎕cself
[1]   ⍝ Ajay Askoolum
[2]   ⎕cself←'c' ⎕cse 'Init' 'System'
[3]   ←⎕cse 'ExecStmt' 'using System;'
[4]   ←⎕cse 'LoadAssembly' 'C:\Windows\Microsoft.NET\Framework\v4.0.30319\System.IO.Compression.FileSystem.dll'
[5]   ⎕cse 'ExecStmt' 'System.IO.Compression.ZipFile.ExtractToDirectory(@{0},@{1});' ZipFileName UnZipDirectory
[6]   ⎕cse 'Close'
[7]   →0
[8]   ⍝ verify path to System.IO.Compression.FileSystem.dll; this file is part of .NET Framework
[9]   ⍝ UnZipDirectory ... must have accesss to location ... last level must NOT exist
[10]  ⍝ ZipFileName must be a valis Zip file
    ∇


I have added some comments to the listings. If interested, please test using typical scenarios and post your feedback. The Zip files created using these functions can be expanded with File Explorer aka Windows Explorer and Zip software.

If there is any interest, I can provide the APLWIN functions for AES256 encryption/decryption of the zip files provided that it is acceptable to have the encryption key in a separate file (which can itself be encrypted using a chosen encryption key so that you do not have to distribute multiple key files).

Personally, I think that it is preferable to have the encryption key outside the zip file for added security; for instance, you can attach the encrypted zip file to an email and send the key separately via a text message.
PS: Doesn't a hanging ← (used to sink output) look really odd?

[vooka]

@Ajay, I would be very interested in seeing your APLWIN functions for AES256 encryption/decryption. Have you posted these anywhere?

Thank you,
Taylor

[Ajay Askoolum]

@Ajay, I would be very interested in seeing your APLWIN functions for AES256 encryption/decryption.

I just saw this today. Alas, a quick search failed to locate my workspace - I'll continue to search in backups & older computer.

[Ajay Askoolum]

Update:

1. I have not found the workspace.
a. because I have not found a way to search inside *.w3 files
b. it turns out that System.IO.Compression.FileSystem.dll does not support encryption or passwords when creating ZIP files (but see 8 below)
c. I've not given up on the search for the elusive workspace

2. I had a quick look at DOTNETZIP
a. It supports AES256 encryption, apparently the same as WINZip AES256
b. It supports per file passwords when multiple files are inside a ZIP it creates

3. I have tested this with APL+Win CSE.

4. Options:
a. Use CSE with the code as a literal array of rank 2.
b. Use CSE with the code packed into a DLL that CSE can load
c. I package the code as a COM server and you
i. register the COM
ii. use the COM with []WI

5. Dependencies:
a. Dot Net Framework 4.0
b. DOTNETZIP.DLL (from the URL above).

6. A ZIP file created with DotNetZip.DLL WITHOUT AES encryption and WITHOUT file password CAN be extracted by Windows (Locate the file in File Explorer, right-click and select Extract All)

7. A ZIP file created with DotNetZip.DLL WITH AES encryption CANNOT be extracted using the method detailed in the 6.

8. A ZIP file created with DotNetZip.DLL WITHOUT AES encryption but WITH passwords CAN be extracted using the method detailed in 6. BUT you will be prompted for the password.

I am not fully acquainted with your requirement; perhaps you can elaborate.

[Davin Church]

Ajay, just FYI --

You can search inside of .w3 files by writing a program to open a second copy of APL as an ActiveX control and then telling it to load each of your workspaces in sequence and look in each one for what you want.

[Ajay Askoolum]

and look in each one for what you want.

How do you (automate) do this from the client session?

[Davin Church]

There's a section in the documentation talking about how to access APL as an ActiveX control. Use []WI 'New' to create an external system, then []WI 'SysCommand' 'LOAD ...' to bring in a workspace, then []WI 'Xcall' to execute a search program of your choice that's in the workspace. Additional calls can be used to run system commands, system functions, and other methods to bring in search tools if they're not already present in the workspace.