[Ann] Zinc HTTP book

Hello Pharoers and others

I’m super happy as editor of the Pharo technology collection to announce the release of a new book on Zinc the excellent and elegant HTTP/S framework for Pharo.

http://books.pharo.org/booklet-Zinc/

I would like to thank Sven Van Caekenberghe for his excellent work. Thanks again Sven for your Pharo support and your great libraries.

 

S. Ducasse

[Ann] New chapter for the Testing in Pharo book

Just a little announce to thank Juan-Pablo Sandoval and his crew for the new chapter on testing Web applications with Pharo.

  TestingInPharo-wip.pdf

S.

[Ann] some communication material …

Hi guys

We designed some communication material around Pharo: sticker, tshirt, flyers,….

https://files.pharo.org/media/

S

[Ann] Quuve Moves to the Public Domain under MIT License

Quuve provides a pro-grade integrated environment for portfolio management and research for securities investors. Professional grade in that it was meant to be licensed out to firms/groups offering research & portfolio management services. It is a virtualized, scaleable, web application developed with Pharo and deployed on GemStone/S. See the Quuve Overview video on YouTube for more information.

Quuve development stopped in the Fall of 2017 due to a lack of funding, however it still has some cutting-edge features for the industry. Because Quuve can function as an institutional level data server, tailored mobile apps could be built as standalone technology – and such an approach could even take advantage of Quuve’s analyst/site model subscription selections.

Software Positives:

  • Using a Glorp-like interface to access data: Quuve has successfully run on Pharo (stand alone for development, backed by Fuel), Gemstone (deployment), and with portions backed using a relational DB – not required nor used at present. Quuve provides DB admin tools including import/export of DB’s in part or full; deleted records access tables, and more. User permission-masks can be granted by DB, table, even patterns in records.

  • Generic CRUD tables UI allows for rapid class development/usage – see ApplicationContext (stack with preferences style inheritance).

  • 3-tier model (application, site, user) for data and Rules, allows site (group) and user specific overrides, which is important in Finance, where few will agree on how to define “free cash flow” in one accounting system, or one industry, let alone globally.

  • Processors allow for dynamic code-based answers to data requests; they blur the lines between data and Rules. Any object can be wrapped with a Processor and have data and rules added dynamically. Rules allow data-requests to perform any operation a modern computer can perform, and most often these are simple math operations on series. Rules are organized by “analyst” names (akin to rules’ Classes), the specifications for analysts is user specific, allowing for dynamic-inheritance required for a pro-grade multi-user Finance application pulling in international data (using different source accounting standards). Rules are compiled on demand; all caching is dynamic and user specific. Rules can be defined using wild-card method-names to open up entire remote volumes of data, or data transformations, with a new pattern. Data series are very robust around missing data and math errors. Quuve includes time profiling and debugging method-trace system to understand data missing or debug errors – enabled in user-specific permissions.

  • Demonstrated integration of learning systems for analysis and portfolio management (“robo-trading”). Data-sampling uses “point in time view” to support accurate backtesting. Portfolios have the unique feature of being able to scroll backwards in time to any date to see the specific holdings there – as models are built from trade-event records.

  • Access to millions of FREE data series from sites like FRED (St. Louis Federal Reserve Bank – USA), World Bank, etc. – menu selectable in report writer!

  • NPI (Non-public information) masking tools – important for privacy in professional Finance office.

  • Script Tool allows for ad hoc testing – most tools like the company report tool also incorporates an ad hoc scripting area.

  • User-specific configurable DB tables access, window access, etc. User and DB bindings specified in CSV files for bootstrapping.

  • Programmer documentation in wiki & in-system help-notes. Some ‘company reports’ are training tools.

Software Issues (“opportunities”!):

  • Quuve development stopped in the Fall of 2017 due to a lack of funding; some dialogs already look a bit out-dated.

  • Some external libraries (e.g. javascript charting) require external licenses – see Credits listed below.

  • Professional grade data must be licensed for individual or larger group – please consult us. Note: we can provide small amounts of CSV data for testing company reports. Perhaps we can form a consortium to license pro-grade data – but we need a Champion to lead the way. CSV data can load on-demand and depends only on the user’s data-source selection via preference or dialog-input specification. Small amounts of data can be sourced from certain sites on an individual basis but not commercially, e.g. yahoo, google, etc.

  • Data loaded currently into one default currency – defined in data-loaders – the idea was to use “table decorators” model to map data to any currency at current exchange ratio or historical.

  • Dropbox was used for file-sharing on cloud servers – not required for local Pharo execution.

  • Quuve configuration has not been updated in 18 months.

  • In recent years only Apple platforms were used for Pharo development.

  • Currently there is minimal support for Quuve. We can add users to our existing programmer wiki and slack-threads to pick up where we left off. In adding users we would prefer a few dedicated champions. There are tools that can be harvested for non-Finance applications. We can add a few trusted users to our current demonstration server, but this contains data that cannot be shared, so we must be selective!

  • Latest working image was Pharo 6.1 32 bits and GemStone 3.4.0

  • Latest instructions to load code are in our devwiki titled DevelopmentBootstrapping_IAM – only a few dedicated users will be added initially.

  • Code is in smallhub and should become public, MIT, and possibly moved to Github. This is an immediate need!

  • GemStone scripts to build new sites, maintenance etc are in gitlab – again, we can start providing access selectively!

  • We have only one server, running in Linode: https://cloud.linode.com/linodes/866838/summary

More info:

Currently copyrighted to Debris Publishing, Inc. – This is changing to MIT License.

Quuve is intended to assist investors manage portfolios and perform research.

Quuve was built and deployed using tools from various sources. The following firms, organizations, communities, and individuals have not endorsed Quuve nor Debris, but we offer due credit whether they are aware of us or not. Many thanks for/to…

Development and deployment environments and platforms

Pharo (c) Pharo.org Pharo is our development platform, language and environment.

GemStone (c) GemTalk Systems We use GemStone system to deploy and run each Quuve site for production. In addition, GemStone serves as our scalable object database.

Web frameworks

Seaside Web Framework The web UI of Quuve is built using Seaside: a great web framework ideal for developing dynamic and complex applications

Twitter Bootstrap for Seaside Bootstrap is very nicely integrated in Seaside web framework and we use it for styling and designing Quuve web components.

Highcharts for Seaside All our web charts use Highcharts JS bindings for Seaside

Magritte Magritte provides us a nice and small meta-description framework which allow us to build a fully featured CRUD system. Most of Quuve forms and reports are automatically generated from our CRUD framework build on top of Magritte.

Development tools and frameworks

Fuel Serializer We use Fuel to save and load our development and testing databases when working in Pharo platform

SIXX Serializer SIXX is used to move small databases from GemStone to Pharo or vice-versa or between different GemStone instances

Metacello To manage software dependencies, releases, environment building, deploying, etc. we use Metacello

Zinc and Zodiac HTTP Components Both are used as HTTP and HTTPS clients. In Addition, Zinc is used as the web server when using Seaside in Pharo

Blowfish We use Blowfish as part of our encryption and decryption tools.

XMLParser We also need to parse and write XML in many places

Javascript Libraries

JQuery and JQuery-UI Even if small, most of our JS development is using JQuery when possible. In addition, a couple of components like autocompletion, popups, sortable lists, etc are used from JQuery-UI

TinyMCE For nicely editing notes, documents, etc. we use the fully WYSIWYG editor TinyMCE

Datatables To improve or HTML tables and reports we use Datatables. This allow us to have fixed header, fixed columns and many many other features.

CodeMirror For scripting, editing accounting and computation rules, and many other places, we use CodeMirror to color highlight code, autocomplete, etc.

Infrastructure

CentOS Linux All our guest/virtual operating systems are CentOS Linux, with latest stable release and security updates.

Nginx Our web server for production purposes is nginx.

Monit We use monit to monitor all processes, being that Quuve ones or the ones we need from the Operating System. Monit will, for example, automatically restart Quuve processes upon crashes.

Fail2ban As part of our security efforts we use fail2ban to protect us from DOS attacks and from malicious user

Firewalld We also use firewalld as out OS firewall

[Ann] New VM promoted to Stable for Pharo 8

Hi,
a new VM has been promoted for Pharo 8. This VM has been used in Pharo 9 during the last 4 weeks without incidents and as the latest one for Pharo 8 during the same period of time.

Basically this VM solves the following main issues:

– GC memory corruption during saving the image
– LibSSH error in Windows.

And a couple of small issues.

The VM is downloaded directly when using zero conf (http://get.pharo.org).
And in new installations of the Pharo Launcher.

The new VM should be from 2020-02-12.
Depending of the Pharo Launcher version, maybe the table is not auto-updated 😛
To check the correct version, close the VM manager window and open it again.
Cheers,
Pablo

[ANN] COVID-19 analysis in Pharo

Hello,
I have set up a new Pharo project for the analysis of COVID-19 outbreak:
Thanks to the ObjectProfile for Roassal and PolyMath contributors for the DataFrame package.
Cheers,
Hernán

DNS over HTTPS (DoH)

Hi,

Today Firefox switched over to using ‘DNS over HTTPS (DoH)’ by default.

https://blog.mozilla.org/netpolicy/2020/02/25/the-facts-mozillas-dns-over-https-doh/

We can do this in Pharo as well, even out of the box (minus the interpretation of the results, but still).

First, what is this ? A good description is:

https://developers.cloudflare.com/1.1.1.1/dns-over-https/

Using the Cloudflare server, we can do the following in Pharo, using the JSON wire format.

ZnClient new
url: ‘https://cloudflare-dns.com/dns-query‘;
accept: ‘application/dns-json’;
queryAt: #name put: ‘pharo.org‘;
queryAt: #type put: ‘A’;
contentReader: [ :entity | STONJSON fromString: entity contents ];
get.

The actual address can be accessed inside the returned result.

SocketAddress fromDottedString: (((ZnClient new
url: ‘https://cloudflare-dns.com/dns-query‘;
accept: ‘application/dns-json’;
queryAt: #name put: ‘pharo.org‘;
queryAt: #type put: ‘A’;
contentReader: [ :entity | STONJSON fromString: entity contents ];
get) at: #Answer) first at: #data).

If you load the following code,

https://github.com/svenvc/NeoDNS

it is just as easy to use the binary UDP wire format.

ZnClient new
url: ‘https://cloudflare-dns.com/dns-query‘;
accept: ‘application/dns-message’;
contentWriter: [ :message |
ZnEntity with: message asByteArray type: ‘application/dns-message’ ];
contentReader: [ :entity |
DNSMessage readFrom: entity readStream ];
contents: (DNSMessage addressByName: ‘pharo.org‘);
post.

Again, the actual address can be accessed inside the returned object.

(ZnClient new
url: ‘https://cloudflare-dns.com/dns-query‘;
accept: ‘application/dns-message’;
contentWriter: [ :message |
ZnEntity with: message asByteArray type: ‘application/dns-message’ ];
contentReader: [ :entity |
DNSMessage readFrom: entity readStream ];
contents: (DNSMessage addressByName: ‘pharo.org‘);
post) answers first address.

Incidentally, a more robust answer can be got as follows:

NeoSimplifiedDNSClient default addressForName: ‘pharo.org‘.

Sven

[Ann] Mediaclue getting open-source

Hi all

I’m a Pharo lover since many years. A huge thanks to all of you, making Pharo possible! As a giving back I would like to open source some of my business projects (Thanks to my supporting customers, too). The first is a web based media assets management called “mediaclue” (Think of it as a image/audio/video/document sharing platform, e.g. for teachers inside a school):

***********************************************
https://github.com/talk-small-be-open/mediaclue
***********************************************

(Everything is ready for simple install into a fresh Linux server with included Ansible provisioning scripts to create the whole appliance)

Cheers!
Andreas

P.S. The UI language is still german at the moment, but that should turn multilanguage eventually

—————————————–
Brot? www.brotrezept.ch!

Andreas Brodbeck
Software-Macher
mindclue GmbH
Dipl. El.-Ing. ETH

+41 55 622 26 24
www.mindclue.ch

[Ann] PolyMath 1.0.2

Dear all,

we are happy to announce the release of PolyMath 1.0.2, the
computational framework for Pharo. This release works on Pharo 7.0 and Pharo 8.0. All the 815 tests and the CI are green on MacOs, Windows and Linux 🙂

Please find more information about the project here:
https://github.com/PolyMathOrg/PolyMath

Changelog of modifications since version 1.0.1 is available here:

https://github.com/PolyMathOrg/PolyMath/milestone/4?closed=1

You can install the 1.0.2 version in a fresh Pharo 7.0&8.0 with the
following code snippet:

Metacello new
repository: ‘github://PolyMathOrg/PolyMath:v1.0.2/src‘;
baseline: ‘PolyMath’;
load

You can join the dev team on #polymath channel for weekly meetings on Thursday. We are looking for more people for tests and contributions.
I’m starting a small experiment. If you want to support me working on PolyMath or others of my project like Tensorflow Pharo bindings, you can be one of my github sponsors.
Please give a monthly support here:
Regards,

Serge Stinckwic

h
Int. Research Unit
 on Modelling/Simulation of Complex Systems (UMMISCO)
Sorbonne University
 (SU)
French National Research Institute for Sustainable Development (IRD)
U
niversity of Yaoundé I, Cameroon
“Programs must be written for people to read, and only incidentally for machines to execute.”
https://twitter.com/SergeStinckwich

Hacking Pharo in Amsterdam

Hacking Pharo in Amsterdam

Come join us in Amsterdam at our programmers workspace in Amsterdam. Its a sea container studio at NDSMTreehouse, open 24/7 (but please make an appointment until you have your own key).

Morphle Inc. T.T. Neveritaweg 57-E1 1033 WB Amsterdam +31507200071 morphle@ziggo.nl