Author Archives: Stéphane Ducasse

Pharo Mooc testimonies

“J’ai trouvé ça très intéressant, beaucoup plus que prévu ! je regrette de ne pas m’y être mis plus tôt. J’ai enfin l’impression de vraiment faire de la POO ! Ou à l’inverse je me rend que je n’en faisais pas vraiment…”

Translated 🙂

“I found it very interesting, much more than I expected! I regret not having started earlier. I finally feel like I’m really doing OOP! Or on the other hand I realize that I wasn’t really doing it…”

Working on a new mooc on advanced design

[Ann] Beta Testing ARM64 Linux & Windows

In the last months, we have been working on the implementation of a MIT-licensed ARMv8 Just-In-Time compiler for Pharo VM. We would like to announce that a beta first version of our JIT backend is available for beta-testing for ARM Linux and Windows machines. 


We are very happy with the advance in this subject, as we have not only implemented a new backend but we have added more than 2500 tests of the JIT, the primitives, code generation, plugins, and the VM in general. Also, we are generating a lot of documentation and improving the process to really democratize the development of the VM. 

Below there are details on how to obtain the versions for testing it.

The following sections give more details of the current status, and the following steps including Apple Silicon, Windows, and Linux ARM64.

Current Status

Our objective is to have a running JIT for the new aarch64 architecture (ARM 64bits). This task includes not only a new backend for the JIT compiler but also adding support for external libraries, dependencies, and the build process. This means having a working VM with comparable features as the one existing in Intel X64. We are targeting all the major operating systems running on this platform (Linux, OSX, Windows). Each of them presents different restrictions and conditions.
This is the current status:

  • We implemented a full backend for the JIT compiler targeting aarch64.
  • All the image side was adapted to run on it, tested on Ubuntu ARM 64 bits.
  • We added support for: Iceberg (Libgit) / Athens (Cairo) / SDL / GTK
  • We implemented a LibFFI based FFI backend as the default one for Pharo 9 in aarch64 (next to come in all platforms). This opens the door to easily port the features to other platforms and OSes. This new implementation does not have an impact on performance as it has comparable results with the old implementation. Much more, this implementation opens also the door to have runners in different threads than the VM thread (non-blocking FFI). 

Status By Platform

Linux

We have a working version of the VM. It is using the same code-base and build process for both architectures (Intel X64 / ARM aarch64). A version can be compiled and executed in all the platforms. Also, we have integrated the build using Open Build Services (OBS) to generate packages for all the mainstream distributions. Currently, the version for Debian-like systems (Debian, Ubuntu, Raspberry pi OS, etc) is available and ready to beta test; the details for installing them are in the end of the mail. Versions for ARCH-based and RPM-based systems are in the working (If you want to get them or to test them please contact us, so we can pass on the detail of how to get them).

Windows

We have extended the build process to fully support Microsoft Visual Studio compilers and more flexibility to select the targets, also we are building it to run in Windows ARM. To correctly run the VM in Windows it is needed to build all dependencies for aarch64. In this path, we have a working Intel x64 version with Cygwin and MSVC, and a MSVC version for aarch64 (ARM64).The ARM64 version is available to be beta-testing on machines with this architecture.The Details are at the end of the mail.

OSX

Our third target is to have a working version for the newest Apple silicon. We are acquiring the corresponding hardware to test and to address the differences in the API exposed to JIT applications. As it is the case of the Windows VM, there is no need to change the machine code generation backend, as it is the same that has been implemented for Linux; but to compile external libraries and particularities of the new OS version.
Thanks for your support. In the meantime, we will continue giving you news about the current state and where are we going.

Beta Testing – Linux

For Debian like distributions, the instructions are the following.

1) Add a new APT source, usually modifying a file in /etc/apt/sources.list.d/, for example, we can add the pharo.list file with the following content. Also, we need to register the key of the repository. These steps should be done only once. We are giving a little script for modifying the configuration. It is only needed to execute line by line.

For Raspbian:

echo “deb https://download.opensuse.org/repositories/devel:/languages:/pharo:/latest/Raspbian_10/ ./” >> pharo.list

sudo cp pharo.list /etc/apt/sources.list.d/pharo.list

wget https://download.opensuse.org/repositories/devel:/languages:/pharo:/latest/Raspbian_10/Release.key

sudo apt-key add Release.key

For Ubuntu:

echo “deb https://download.opensuse.org/repositories/devel:/languages:/pharo:/latest/xUbuntu_20.04/ ./” >> pharo.list

sudo cp pharo.list /etc/apt/sources.list.d/pharo.list

wget https://download.opensuse.org/repositories/devel:/languages:/pharo:/latest/xUbuntu_20.04/Release.key

sudo apt-key add Release.key

2) Update the repository to take the current versions

sudo apt update

3) Install Pharo9 VM

sudo apt install pharo9 libgit2

4) In case of using the interactive version (with UI) check if the packages libsdl2-2.0-0  and libcairo2 are installed (Usually they are installed, but depends what it is installed in the system). These are not needed for the headless execution or if it is not used by the image.

Getting the images:

The images can be downloaded using zero-conf, the latest image is ready to be used. It can be done with:

wget -O – get.pharo.org/64/90 | bash

Running Pharo

pharo Pharo.image eval 42 factorial

Or interactive

pharo Pharo.image –interactive

Windows Testing

A version of the VM is ready to be tested in Windows ARM64. A zip containing the VM is available in:
https://files.pharo.org/vm/pharo-spur64/Windows-ARM64/PharoVM-9.0.0-4df0e562a-Windows-ARM64-stockReplacement-bin.zip

As in Linux, the latest image is compatible with this version of the VM.

Error reporting

In case of encountering errors please report them attaching:

 – crash.dmp file and PharoDebug.log
 – Version of the image using (in case of any community project loaded, the names of those)

 – Complete machine version (the result of executing uname -a)
 – Complete information about the Linux distribution

 – Steps for reproducing the error (if needed)

Thanks for the effort!!
— 
Pablo Tesone.
tesonep@gmail.com

Understanding Base58 Encoding Medium article

Hi,

I wrote and published another article in Concerning Pharo:

Understanding Base58 Encoding

It is all about integers.

This introductory article shows a bit how easy and how much fun it is to do integer and binary encoding and decoding as well as printing and parsing in Pharo. It also tries to explain Base58 Encoding from a first principles point of view.

Sven

Roassal on ARM64 windows

Guille P. just sent us the following screenshot showing Roassal on ARM64 Windows….

This implies that Athens and Cairo are building correctly :).

ARM Rasp first positive feedback

As doing is better than many hand waving we want to share with you the following information.

Hi,

This is fantastic news on several levels – the new JIT itself, but even more the new approach, process, infrastructure and tests.

(Sorry, but this is a long/technical mail).

A week ago I was one of the first people outside the development team to be able to test the new ARM64 JIT VM on hardware they did not even test on.

In particular I used an Amazon AWS EC2 T4g.micro instance (1 GB) with Ubuntu Server 20.04.1 LTS.

These machines use an ARM64 CPU (AWS Graviton2, Neoverse N1, Cortex-A76, ARM v8).

ubuntu@ip-172-30-0-23:~/test$ uname -a
Linux ip-172-30-0-23 5.4.0-1030-aws #31-Ubuntu SMP Fri Nov 13 11:42:04 UTC 2020 aarch64 aarch64 aarch64 GNU/Linux

ubuntu@ip-172-30-0-23:~/test$ lscpu 
Architecture:                    aarch64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
CPU(s):                          2
On-line CPU(s) list:             0,1
Thread(s) per core:              1
Core(s) per socket:              2
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       ARM
Model:                           1
Model name:                      Neoverse-N1
Stepping:                        r3p1
BogoMIPS:                        243.75
L1d cache:                       128 KiB
L1i cache:                       128 KiB
L2 cache:                        2 MiB
L3 cache:                        32 MiB
NUMA node0 CPU(s):               0,1
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:        Mitigation; __user pointer sanitization
Vulnerability Spectre v2:        Not affected
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
Flags:                           fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp ssbs

My reaction after one hour ?

Wow, wow, wow, this is incredible. It all just works and it seems pretty fast as well. I played with the vm/image for a couple of minutes and so far everything worked as expected and I had no crashes at all.

The order of magnitude of 1 tinyBenchmarks is very similar to other (server) machines:

"'1894542090 bytecodes/sec; 146296146 sends/sec'" "arm64"
"'2767567567 bytecodes/sec; 258718969 sends/sec'" "macOS"
"'1227082085 bytecodes/sec; 109422120 sends/sec'" "aws"
"'2101590559 bytecodes/sec; 166532391 sends/sec'" "t3 lxd"

Here is a benchmark in the HTTP space, how fast can ZnServer respond to multiple concurrent requests over the local network:

$ ./pharo Pharo.image eval --no-quit 'ZnServer startDefaultOn: 1701' &
$ ab -k -n 1024 -c 8 http://localhost:1701/small
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Completed 102 requests
Completed 204 requests
Completed 306 requests
Completed 408 requests
Completed 510 requests
Completed 612 requests
Completed 714 requests
Completed 816 requests
Completed 918 requests
Completed 1020 requests
Finished 1024 requests
Server Software:        Zinc
Server Hostname:        localhost
Server Port:            1701
Document Path:          /small
Document Length:        124 bytes
Concurrency Level:      8
Time taken for tests:   0.268 seconds
Complete requests:      1024
Failed requests:        0
Keep-Alive requests:    1024
Total transferred:      317440 bytes
HTML transferred:       126976 bytes
Requests per second:    3814.45 [#/sec] (mean)
Time per request:       2.097 [ms] (mean)
Time per request:       0.262 [ms] (mean, across all concurrent requests)
Transfer rate:          1154.76 [Kbytes/sec] received
Connection Times (ms)
             min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:     0    2  19.0      0     267
Waiting:        0    2  19.0      0     267
Total:          0    2  19.0      0     267
Percentage of the requests served within a certain time (ms)
 50%      0
 66%      0
 75%      0
 80%      0
 90%      0
 95%      0
 98%      0
 99%     42
100%    267 (longest request)

That is 3800 req/s with 8 concurrent threads, each response 124 bytes. And the output document is dynamically generated each time !

Now a cached static binary document, first a small one (64 bytes):

$ ab -k -n 1024 -c 8 http://localhost:1701/bytes
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Completed 102 requests
Completed 204 requests
Completed 306 requests
Completed 408 requests
Completed 510 requests
Completed 612 requests
Completed 714 requests
Completed 816 requests
Completed 918 requests
Completed 1020 requests
Finished 1024 requests
Server Software:        Zinc
Server Hostname:        localhost
Server Port:            1701
Document Path:          /bytes
Document Length:        64 bytes
Concurrency Level:      8
Time taken for tests:   0.214 seconds
Complete requests:      1024
Failed requests:        0
Keep-Alive requests:    1024
Total transferred:      256000 bytes
HTML transferred:       65536 bytes
Requests per second:    4778.62 [#/sec] (mean)
Time per request:       1.674 [ms] (mean)
Time per request:       0.209 [ms] (mean, across all concurrent requests)
Transfer rate:          1166.65 [Kbytes/sec] received
Connection Times (ms)
             min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:     0    2  12.3      0     207
Waiting:        0    2  12.3      0     207
Total:          0    2  12.3      0     207
Percentage of the requests served within a certain time (ms)
 50%      0
 66%      0
 75%      0
 80%      0
 90%      0
 95%      0
 98%      5
 99%     64
100%    207 (longest request)

That is 4700 req/s

Now a larger one, 1024 bytes:

$ ab -k -n 1024 -c 8 http://localhost:1701/bytes/1024
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Completed 102 requests
Completed 204 requests
Completed 306 requests
Completed 408 requests
Completed 510 requests
Completed 612 requests
Completed 714 requests
Completed 816 requests
Completed 918 requests
Completed 1020 requests
Finished 1024 requests
Server Software:        Zinc
Server Hostname:        localhost
Server Port:            1701
Document Path:          /bytes/1024
Document Length:        1024 bytes
Concurrency Level:      8
Time taken for tests:   0.228 seconds
Complete requests:      1024
Failed requests:        0
Keep-Alive requests:    1024
Total transferred:      1241088 bytes
HTML transferred:       1048576 bytes
Requests per second:    4484.93 [#/sec] (mean)
Time per request:       1.784 [ms] (mean)
Time per request:       0.223 [ms] (mean, across all concurrent requests)
Transfer rate:          5308.34 [Kbytes/sec] received
Connection Times (ms)
             min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:     0    2  16.2      0     227
Waiting:        0    2  16.2      0     227
Total:          0    2  16.2      0     227
Percentage of the requests served within a certain time (ms)
 50%      0
 66%      0
 75%      0
 80%      0
 90%      0
 95%      0
 98%      0
 99%     41
100%    227 (longest request)

Still 4400 req/s – 1024 requests finished in about 0.25 seconds, transferring 1Mb.

These are really good numbers !

And under this load, the image+vm remained totally stable.

Great, great work and thanks again everyone for the effort. You can be very proud for this achievement.

Sven

Call for Beta-testers Pharo ARM64 JIT

Dear happy Pharoers, 
Over the last months we have been working on the implementation of a MIT-licensed ARMv8 Just-In-Time compiler for Pharo VM.We are very happy with the advance on this subject, as we have not only implemented a new backend but we have added more than 2500 tests to the JIT, the primitives, code generation, plugins and the VM in general. In the process we fixed many problems. It shows that following the roadmap that was decided during General assembly of PharoDays, we have been investing in the Pharo VM and that our efforts are greatly paying off. Also, we are generating a lot of documentation and improving the process to really democratize the development of the VM. So Pharo is a better shape than ever on such matter and this opens a lot of possibility for the future. 


## Call for Beta-Testers


We would like to announce that a first version of our JIT backend is available for beta-testing for ARM Linux machines using Ubuntu. We are now entering into a beta testing stage of the VM and the image in Ubuntu ARM 64.We would like to invite you to our beta testing phase for the VM. If you’re interested in participating, please contact pablo.tesone@inria.fr.
The following sections give more details of the current status, and the following steps including Apple Silicon, Windows ARM64 and Linux Open Build System support.


## Current Status

Our objective is to have a running JIT for the new aarch64 architecture (ARM 64bits). This task includes not only a new backend for the JIT compiler but also adding support for external libraries, dependencies and the build process. This means having a working VM with comparable features as the one existing in Intel X64. We are targeting all the major operating systems running in this platform (Linux, OSX, Windows).Each of them present different restrictions and conditions.
This is the current status:

  • We implemented a full backend for the JIT compiler targeting aarch64.- All the image side was adapted to run on it, tested on Ubuntu ARM 64 bits. 
  • We added support for: Iceberg (Libgit) / Athens (Cairo) / SDL / GTK
  • We implemented a LibFFI-based FFI backend as the default one for Pharo 9 in aarch64 (next to come in all platforms). This opens the door to ease port the features to other platforms and OSes. 

## Following Steps and Open Betas: Linux Open Build System (OBS), Windows ARM64 and Apple Silicon

Linux Systems: In the following days, we will also support Raspbian (Debian) and Manjaro on ARM64. For doing so, we are pushing the last details in having a single Linux build system through OBS. So, if you want to start doing beta-testing of these versions please contact us. A public beta will be open in around two weeks.

Windows Systems: We have extended the build process to fully support Microsoft Visual Studio compilers and more flexibility to select the targets, also we are building it to run in Windows ARM. To correctly run the VM in Windows it is needed to build all dependencies for aarch64. In the following weeks, we expect to have a working Non-JIT version and a JIT version. The remaining points to have a JIT version are related with the build process as the API of the operating system has not changed from X64 to aarch64.

OSX Systems: Our third target is to have a working version for the newest Apple silicon. We are acquiring the corresponding hardware to test and to address the differences in the API exposed to JIT applications. As it is the case of the Windows VM, there is not need to change the machine code generation backend; but to compile external libraries, and particularities of the new OS version.

Thanks for your support, and again, if you like to start beta testing the VM please contact us. In the meantime, we will continue giving you news about the current state and where are we going. 
The consortium would like to particularly thank Schmidt Buro and Lifeware for their contracts. 
Regards,
Pablo in behalf of the Pharo Consortium Engineers

[Fr] Pharo quoi de neuf ?

We want to thanks tissac and colleagues for this nice article on Pharo for Linux.fr

https://linuxfr.org/news/pharo-quoi-de-neuf

S

[Ann] Dophin to Pharo

Rko281 (alias John Aspinall) announced a nice package to migrate code from Dolphin to Pharo

https://github.com/rko281/DolphinToPharo

rko281 is also the author of the nice restore Database system.

https://github.com/rko281/ReStoreForPharo

We love these packages that support business. Go go go!

S.

[Ann] Symbolic Algebra package

I am happy to announce that I’ve finished a stable version of a new package / library. All started when I wanted a Calculator in Pharo, for which I needed a parser, and then I needed to store Mathematical expressions, and variables, and constants, and precedence and simplification and other hundred things. Eventually became a small Symbolic Algebra package. The name is Mathemagics. Has the ability to either build expressions manually with messages, or via a built-in parser. It can simplify an expression (such as x*x = x^2), supports Number-like methods, and even a bit of Calculus generating derivatives.

Parse via expression, simplify, sort terms, use symbols (pi => π)

  | fx |
  "Notice that constants like pi display a symbol (both input supported)"
  fx := MathExpression from: 'x*x - (pi * -1) + x*8/1'.
  fx simplify.                                                  "x^2 + 8·x + π"

Derivatives

  | fx dFx |
  fx := MathExpression from: 'x*x + ln(x)'.
  dFx:= fx derivativeFor: 'x'.                                  "2·x + 1/x"

Block generation

  "Expression to a Smalltalk BlockClosure"
  | fx x |
  "Math precedence means no parenthesis required"
  fx := MathExpression from: 'x + 3 * pi'.                      "x + π·3"

  "New method will generate code with Smalltalk precedence"
  fx asBlock                                                    "[ 😡 | x + (3 * Float pi) ]"

It comes a small footprint Parser, with a help on Pharo Help, and with the Calculator added to the menu. Here is the GitHub page for download. Please let me know your opinion. Has involved lots of hours, hope is useful to many.

https://github.com/grpistoia/Mathemagics

Parse via expression, simplify, sort terms, use symbols (pi => π)

  | fx |
  "Notice that constants like pi display a symbol (both input supported)"
  fx := MathExpression from: 'x*x - (pi * -1) + x*8/1'.
  fx simplify.                                                  "x^2 + 8·x + π"

Derivatives

  | fx dFx |
  fx := MathExpression from: 'x*x + ln(x)'.
  dFx:= fx derivativeFor: 'x'.                                  "2·x + 1/x"

Block generation

  "Expression to a Smalltalk BlockClosure"
  | fx x |
  "Math precedence means no parenthesis required"
  fx := MathExpression from: 'x + 3 * pi'.                      "x + π·3"

  "New method will generate code with Smalltalk precedence"
  fx asBlock                                                    "[ 😡 | x + (3 * Float pi) ]"

Gus on discord

PHARO VM RUNNING ON Rpi4 AARCH64

Rpi4 AARCH64 “‘727789623 bytecodes/sec; 49229431 sends/sec'”

In Amazon ARMv8 Machine[5:33 PM]

ubuntu@ip-172-31-56-119:~/dev/pharo/image$ uname -a && ../vm/pharo --version && ../vm/pharo Pharo.image eval 1 tinyBenchmarks
Linux ip-172-31-56-119 5.4.0-1029-aws #30-Ubuntu SMP Tue Oct 20 10:08:09 UTC 2020 aarch64 aarch64 aarch64 GNU/Linux
Pharo 9.0.0 built on Nov 25 2020 16:08:45 Compiler: 9.3.0
Built from: v8.1.0-alpha-1107-gdc0334322 - Commit: dc0334322 - Date: 2020-11-18 19:04:24 +0100
'1880624426 bytecodes/sec; 142130770 sends/sec'
ubuntu@ip-172-31-56-119:~/dev/pharo/image$

Really nice Pablo!!!