Monthly Archives: November 2018

Pharo IoT hackathon!

Advertisements

Better management of encoding of environment variables

Hi all,

Thanks Ben for reading.

For those wanting a follow up, I’ve proposed this pull request:
https://github.com/pharo-project/pharo/pull/1980.
I’m still working on avoiding dependencies against UFFI, fixing one other
test.
This is however almost finished, and given that I had to adapt the
original *abstract
proposal* to fit the real system, here is an updated version:

API Proposal for OSEnvironment and friends
=========================

OSEnvironment is the common denominator for all platforms. They should
implement at least the following messages with the following semantics:

– *at: aVariableName [ifAbsent:/ifAbsentPut:/ifPresent:ifAbsent:]*

Gets the String value of an environment variable called `aVariableName`.
It is the system reponsibility to manage the encoding of *both arguments
and return values*.

– *at: aVariableName put: aValue*

Sets the environment variable called `aVariableName` to value `aValue`.
It is the system reponsibility to manage the encoding of *both arguments
and return values*.

– *removeKey: aVariableName*

Removes the environment variable called `aVariableName`.
It is the system reponsibility to manage the encoding of *both arguments
and return values*.

API Extensions for *Nix Systems (OSX & Linux)
=========================

Since *Nixes environment variables are binary data that could be encoded in
any encoding, the following methods provide more flexibility to access such
data in the encoding of the choice of the user, or even in binary form.

– *at: aVariableName encoding: anEncoding
[ifAbsent:/ifAbsentPut:/ifPresent:ifAbsent:/put:] / removeKey:**
aVariableName
encoding: anEncoding*

Variants of the common API from OSEnvironment.
The encoding used as argument will be used to encode/decode *both arguments
and return values*.

– *rawAt: anEncodedVariableName encoding: anEncoding
[ifAbsent:/ifAbsentPut:/ifPresent:ifAbsent:/put:] / removeRawKey:*
*anEncodedVariableName*

Variants of the common API from OSEnvironment.
These methods assume arguments and return values are encoded/decoded by the
user, so no marshalling or decoded is done by it.

Rationale
=========================

– Encoding/Decoding should be applied not only to values but to
variables names too. In most cases Ascii overlaps with utf* and Latin*
encodings, but this cannot be simply assumed.
– Windows requires calling the right *Wide version of the functions from
C, plus the correct encoding routine. This could be implemented as an FFI
call or by modifying the VM to do it properly instead of calling the Ascii
version.
– Unix FileSystems and environment variables could mix strings in
different encodings, thus the flexibility added by the low level *Nix
extensions.

Other Implementation Details
=========================

– VM primitives returning paths Strings should be carefuly managed to
decode them, since they are actually C strings (so byte arrays) disguised
as ByteStrings.
– Similar changes had to be applied to correctly obtain the current
working directory in case it is a wide string.

 

Guille

New random generator: WELL512 PRNG

Hi,

By accident I came across a pseudo random number generator that I never heard off before. It is supposed to be pretty good and had a very simple implementation. So I ported it to Pharo.

<class comment>

I am RandomWELL512, a random number generator.

I use the PRNG (Pseudo Randon Number Generator) WELL (Well equidistributed long-period linear) with a 512 bit state.

  https://en.wikipedia.org/wiki/Well_equidistributed_long-period_linear
  http://www.iro.umontreal.ca/~lecuyer/myftp/papers/wellrng.pdf

Implementation algorithm (See #nextUInt32)

  http://www.lomont.org/Math/Papers/2008/Lomont_PRNG_2008.pdf (Chris Lomont, www.lomont.org)

Usage

  RandomWELL512 new in: [ :r | (1 to: 10) collect: [ :i | r nextUInt32 ] ]. 

  RandomWELL512 new useUnixRandomGeneratorSeed; in: [ :r | (1 to: 10) collect: [ :i | r next ] ]. 

  RandomWELL512 new in: [ :r | (1 to: 10) collect: [ :i | r nextInt: 1000 ] ].

  RandomWELL512 new in: [ :random | | count all |
    random useUnixRandomGeneratorSeed.
    count := 1024 * 1024.
    all := Array new: count streamContents: [ :out |
      count timesRepeat: [ out nextPut: random next ] ].
    { all min. all max. all average. all stdev } ].

  [ RandomWELL512 new in: [ :random | 1 to: 1e6 do: [ :i | random next ] ] ] timeToRun.

Note that you should create one instance, seed it properly, and keep using it.

</class comment>

It is acceptably fast, generating 1M [0,1) Floats in about 0.1s. I compared the output with a fixed initial state to the C code that I started from and I got the same numbers. Maybe some people find this interesting.

I attached a file out.

Sven

Zn nicer API

Hi,

I added a new convenience method to Zinc HTTP Components: ZnClient>>forJsonREST. This configures a ZnClient (HTTP client) to talk to standard JSON REST web services. Here are a couple of examples:

ZnClient new
  forJsonREST;
  get: 'https://jsonplaceholder.typicode.com/users'.

What #forJsonREST does is 3 things: set the 'Accept' header to 'application/json', install a #contentReader that parses incoming JSON as well as a #contentWriter that generates JSON.

ZnClient new
  systemPolicy;
  forJsonREST;
  url: 'https://jsonplaceholder.typicode.com/posts';
  contents: { #foo->1. #bar->2 } asDictionary;
  post.

As you can see, the full ZnClient API can be combined when needed.

ZnClient new
  forJsonREST;
  post: 'https://jsonplaceholder.typicode.com/posts' 
  contents: (NeoJSONObject new foo: 1; bar: 2; yourself).

#post:contents: combines separate #url: #contents: and #post message.

#forJsonREST uses NeoJSON[Object|Writer] if found, else STONJSON. If both are missing, this results in an error.
	
ZnClient new
  systemPolicy;
  forJsonREST;
  url: 'http://easy.t3-platform.net/rest/geo-ip';
  queryAt: #address put: '81.83.7.35';
  get.

Finally, here is a more sophisticated example, doing a DNS request over HTTPS:

ZnClient new
  systemPolicy;
  beOneShot;
  forJsonREST;
  accept: 'application/dns-json';
  url: 'https://cloudflare-dns.com/dns-query';
  queryAt: #name put: 'stfx.eu';
  queryAt: #type put: #AAAA;
  get.

Note that in most cases, you will configure one client to a specific endpoint and keep on reusing it. At one point in time it might be good to #close the client (although that happens on finalise as well). For single requests, you can use #beOneShot.

All this can be found in #bleedingEdge (HEAD). There are unit tests as well.

Sven

[Ann] v1.4.0 of Iceberg

Hello!

This week we are releasing the version v1.4.0 of Iceberg.
(https://github.com/pharo-vcs/iceberg/releases/tag/v1.4.0)

This version is available in the latest Pharo 7.

This release fixes a bug introduced in v1.3. It also add new features
in the repository view, add some cleaning and also re-introduce a
feature that was lost.

Thanks to all contributors.

Enjoy!

Full changelog:

Bugfixes

#1068 'There is no associated repository configured.' warning on right
clicking missing repository

Features

#1077 Repository view: Allow to collapse branches/remotes/tags trees
#847 Move tags under remotes in Repository view
#1070 set upstream if missing

Cleanups

#1066 Pharo 7: PackageManifest subclasses should be packaged with "Manifest"
#1015 Replace usages of Glamour in the Github Plugin
#1063 1061-Introduce-iconNamed-in-IceDefinition-and-IceTipModel-and-remove-all-the-terrible-Smalltalk-ui-icons

[ann] Pharo 7.0.0-rc1

Greetings!

I’m announcing today we reach Pharo 7.0.0-rc1!

This is the first step to release a definitive version, and while we will continue integrating bug fixes, API change Pull Requests will be delayed until the open of Pharo 8.0.0 development.
Now, you would wonder what is the ChangeLog of this release… and answer is we still do not have one (btw, we should find a way to automate this).

Anyway… we are very close to release now 🙂

Please download, test, report issues.

[Ann] release v1.3 of Iceberg

Hello!

This week we are releasing the version v1.3 of Iceberg.
(https://github.com/pharo-vcs/iceberg/releases/tag/v1.3.0
+https://github.com/pharo-vcs/iceberg/releases/tag/v1.3.1)

This version will be available after we merge this PR:

https://github.com/pharo-project/pharo/pull/1951

This release contains some new features such as the support of self
hosted gitlab, integration with github, etc.
It also contains multiple bug fixes, cleanups and enhancements.

On the CI part, Guille made the Appveyor build green! This will
increase the stability of the windows support.

Thanks to all contributors.

Enjoy!

Full changelog:

Features

#1021 Self hosted gitlab support
#1027 Improved new tag dialog to generate semantic versionning tags
#1044 Show a button “View on Github” when creating a PR
#1008 Add “create branch from GitHub issue” option
#1048 Add commands to open remotes on Github
#1010 Add menu entry in extras to force calculate diff

Bug corrections

#975 Metacello asks too many times what to install when there are
conflicting versions
#980 Iceberg should Identify better the packages and the normal files
#982 The Edit Project should have a Warning if it will affect the packages
#986 Iceberg does not realize changes in extended classes
#999 Pulling and pushing to a gitolite server asks password
#984 Conversion to Tonel generates corrupted .properties
#1041 Filter in repository view don’t work with capital letters
#1019 Metacello Integration leaves Monticello leftover repositories
#859 Creating a branch and pushing does not sets the upstream
#1043 Packages starting with lowercase not recognized
#991 Error on right click in the project editon wizard
#775 Reviewing a PR is broken
#1036 Debugger if we try to merge without selecting a branch
#1064 Fix failing tests regarding clean code in Pharo

Enhancements

#988 Iceberg should load the packages in a single MCLoader (This will
make the loads of packages atomic)
#1001 Use “instance creation” instead of “instance-creation” for
method protocol name
#1004 Use displayScaleFactor in UI
#977 Add ToolTip help to the Commands
#1030 Better support for binary files
#1034 SSH passphrase is now hidden

Cleanups

#1018 Iceberg UI relies on deprecated classes from Spec and Commander
#1051 Clean useless huge hierarchy in Github plugin UI

Infrastructure Enhancements

#1023 Fix CI for windows

[Pharo-dev] [ANN] The STON Specification

Hi,

Since there can never be enough documentation I finally took some time to write a more formal description of STON as a data format.

  https://github.com/svenvc/ston/blob/master/ston-spec.md

The idea is to let this stabilise a bit and to then update the two other documents describing STON, where necessary:

  https://github.com/svenvc/ston/blob/master/ston-paper.md
  https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessfulBuild/artifact/book-result/STON/STON.html

Also, the latest changes in STON have to make their way to the Pharo image as well.

  https://github.com/svenvc/ston

All feedback is welcome.

Sven