Crystal 0.35.0 released!

Crystal 0.35.0 has been released!

It seems everybody wanted to jump in and polish some corner of the std-lib before 1.0. There was a lot of activity iterating in some recent additions and more cleanups. Also, there were a lot of improvements on debugging, Windows support and stabilization on other platforms runtime.

This will be the last 0.x release. Get ready for some 1.0.0-preX releases.

There are 242 commits since 0.34.0 by 38 contributors.

Let’s get right into some of the highlights in this release. They are plenty. Don’t miss out on the rest of the release changelog which has a lot of valuable information.

Language changes

Exhaustive Case (take 2)

In the previous release we allowed the compiler to check the exhaustivity of the case conditions. From the feedback received, we decided to:

Allow the case ... when to be as before: there is an implicit else nil, hence the conditions can be non-exhaustive.
Introduce the case ... in statements as experimental: they don’t have an implicit else, and the conditions need to be exhaustive or the code will not compile. Experimental means that it’s subject to change based on feedback. Even between minor releases.

This decision stays closer to a more familiar case ... when semantics and will allow further iteration on the exhaustive case constructs without affecting existing code. Read more at #9258 and #9045.

# Compiles! Totally fine
case 1 || "a"
when Int32
end

# Error: missing case String
case 1 || "a"
in Int32
end

To make this change, we needed to make in a proper keyword. Although it is a breaking-change, in was already used in the macro sublanguage as a keyword.

Compiler

We are making a breaking-change in the compiler CLI to make it more comfortable for shebang #! scripts. From now on when running the compiler with a file argument instead of a command like crystal path/to/file.cr arg1 arg2 the file will be compiled, and then executed with the arguments. This means the arguments affect only the program to run and not the compiler.

If you need to use compile flags and want to execute the program with some arguments you can still use the run command: crystal run path/to/file.cr -Dcompiler_flag --release -- arg1 arg2. The -- will split compiler options from program options. Read more at #9171.

Another breaking-change use for some shell integrations is that crystal env will now quote the values. This means it’s safe to eval "$(crystal env)". The crystal env VARIABLE is still the same. Read more at #9428

The parser got a lot of love. Fixing edge cases, bugs and even a considerable refactor in #9208.

Some features of the language needed to iterate to work better together. In this release the time came for auto-casting regarding multi-dispatch and regarding default values against unions. Read more at #9004 and #9366. This should solve unfortunate surprises in some scenarios.

Beyond the improvements of existing features, probably one long awaited feature concerns improvements in Crystal’s debugging capabilities. The story is not yet complete, but in #8538 a huge step forward was made. You can check out an article about how to debug a Crystal program in VS Code article for more information, configuration and screenshots.

The @[Link] annotation was slightly redesigned. It has a better integration with pkg-config and we dropped the static: option. This will allow us to focus on future stories for tweaking linking, yet providing good defaults. Read more at #8972.

For crystal package maintainers, it is worth noticing that when building the compiler the CRYSTAL_CONFIG_PATH only needs the path of the std-lib. The lib directory is now always included by the compiler. This will allow future stories for tweaking the shards installation path. Read more at #9315.

Another change that can impact packages is that the SOURCE_DATE_EPOCH environment variable can be used while building the compiler to advertise the source date. Read more at #9088.

Shards

Shards v0.11.1 is bundled in this release.

The main thing you need to know is that the crystal: property is effectively mandatory now. It is used to filter which versions of a shard are available based on the crystal environment. You can check the semantics in shards/SPEC.md.

To keep backward compatibility if the crystal: property in the shard.yml is missing, it will be interpreted as < 1.0.0. So everything will keep working until Crystal 1.0.0. Yet, if this is inconvenient you can pass --ignore-crystal-version to avoid the check entirely.

We believe that dependency versions need to be stated clearly for better expectations on consumers. The std-lib and language version is also a dependency.

The semantics of the crystal: property though is slightly different from dependencies’ version: crystal: x.y.z is interpreted as ~> x.y, >= x.y.z (ie: >= x.y.z, < (x+1).0.0) for convenience. The result is that on every major release there will be some maintenance burden.

We recommend running shards install on your current project. You will notice that the shard.lock has a new version format with some additions. And there will be a lib/.shards.info file that will describe the installed dependencies. This new file does not need to be tracked.

Finally a new feature is that dependencies allow you to express intersections like version: >= 1.0.0, < 2.0.

For a comprehensive list of changes in this shards version you can check its changelog.

Standard library

You will see many breaking-changes in this release. Most of them have deprecation warnings as usual. We didn’t want to go to 1.0 without cleaning up a lot of stuff beforehand. The cleanup probably should have happened before. It’s been hard to prioritize, but we are getting to it.

Many methods have an IO argument that is a sink or target of the computation. In #5916 you can read the motivation to standardize them as the first argument in these functions. It got implemented in #9134 and some follow up PRs.

Related to all these IO methods, on String, instead of returning String values, there are new overloads to emit directly the result to the IO; read more in #9236. This affects #underscore, #titleize, #capitalize, #upcase, and #downcase methods.

Keep an eye on #9291 for a proposal to improve the interpolation of strings so they can emit directly to an IO.

Another breaking-change in IO is that #skip, #write, #write_utf8, #write_byte, #write_bytes, and #skip_to_end return the number of bytes it skipped/written. This is similar to what other languages do, and serves to account for the position in the stream while writing in it, without additional calls. Read more at #9233 and #9363.

We are introducing the @[Experimental] annotation to mark which parts of the std-lib, language, or shard should be used with extra care. An experimental feature is allowed to change, break, or disappear despite the semver guarantees. For now, the annotation is used in the documentation generator tool. We have a few draft ideas to give more formal semantics to it. Read more at #9244.

You might notice that the Digest types got some refactors and small method renames. Read more at #8426.

Speaking of features, the OptionParser is now allowed to define sub-commands. Read more at #9009.

Some other clean-ups: Flate, Gzip, Zip, and Zlib were moved inside Compress module in #8886. Flate was renamed to Compress::Deflate, actually. You might need to require "compress/gzip" and change some constants here or there. The require "gzip" is still available but will show a deprecation warning.

Some efforts were dedicated toFile & FileUtils, to clean them up and to ensure operations are available from both APIs. Read more at #9175.

Macros

If you are into cross-compiling you’ll be quite happy with the introduction of host_flag? macro method. Similar to flag? but it resolves on the host machine. Read more at #9049.

Numeric

The overflow detection was fixed to correctly handle operations with mixed sign operands. Read more at #9403.

We added Int#digits but along the way it was mandatory to reverse the output of BigInt#digits for consistency. Read more at #9383.

Serialization

The JSON.mapping and YAML.mapping migrated to their own packages: github:crystal-lang/json_mapping.cr and github:crystal-lang/yaml_mapping.cr. They served well but in the presence of JSON::Serializable and YAML::Serializable it’s better to remove them from the std-lib. Read more at #9272.

Time

From now on the default precision of Time#to_rfc3339 will be seconds, without fractions. You can use the fraction_digits named argument to choose between 0, 3, 6, 9 precision digits. In #9283 we are dropping the logic to show the fraction of seconds depending on the time value.

Networking

We’ve updated the SSL server secure defaults in #9026. And fixed some HTTP::Server sporadic failures during SSL handshake in #9177.

There are a couple of breaking-changes in HTTP::Server though. We improved the error handling and logging in #9115 and integrated it with the new logging module. The HTTP::Request#remote_address type was changed to Socket::Address? in #9210.

Logging

First of all, huge thanks for all the feedback and early adopters of the new logging module introduced in 0.34.0. There are several changes that, though they are indeed breaking-changes, do not affect the main APIs. Together they bring additional functionality and improved performance in some use cases.

We renamed Log::Severity::Warning to Warn in #9293. Log.warn { ... } was, and still is, the way to emit a warning. This change affects the :warning and configuration via environment variables mostly. Similarly, we dropped Verbose, and added Trace and Notice in #9107.

The setup of logging got simpler. There are a couple of Log.setup* methods. Each of them will always set up the binding fully between sources and backends.

Log.setup :debug # will show debug or above in the stdout for all source
Log.setup "db.*", :trace # will show trace or above in the stdout for db.* sources and nothing else
Log.setup_from_env # will grab the value of LOG_LEVEL env variable

You might notice that Log.setup_from_env is now using a single environment variable as input. More flexibility will come later, but the new named arguments should offer a better experience. Read more at #9145.

Each entry could already have context information that is grabbed from the running fiber. We essentially split Log::Context responsibilities between Log::Metadata and Log::Metadata::Value. The former is a hash-like structure of Symbol to Log::Metadata::Value with some allocation and algorithmic optimizations. The main work was done in #9227 and #9295. These refactors also drop the immutability guarantee in the Log::Metadata::Value that was achieved via cloning.

One wanted feature that this enables is the possibility to attach local metadata or structured information to a log entry. That is, without the penalty of changing and restoring the context of the current fiber. We were allowed to do this while keeping the initial design of avoiding the creation of values if the entry is not to be emitted.

Log.info { "Program started" }
Log.info &.emit("Program started") # same as previous
Log.info &.emit("User logged in", user_id: 42) # local data
Log.info &.emit("User logged in", expr_that_computes_hash_named_tuple_or_metadata)
Log.warn exception: e, &.emit("Oh no!", user_id: 42) # with exception
Log.warn exception: e, &.emit("Oh no!") # with exception, no local data
Log.warn(exception: e) { "Oh no!" } # same as previous
Log.info &.emit(action: "log_in", user_id: 42) # empty message

How to create custom log formatters was revisited in #9211. Creating a formatter from a block or proc is still an option, but check in some simplified ways to define a formatter from a string directly.

If you want to test that log entries are emitted you can use the new spec helper Log.capture. Read more at #9201.

Concurrency

We dropped Concurrent::Future and top-level methods delay, future, lazy. If you want to keep using them, use the github:crystal-community/future.cr shard. Read more at #9093.

Another feature that was dropped is the parallel macro in #9097.

We expect to develop a more robust approach to cover these scenarios post 1.0.

Runtime

We deprecated Process#kill in favor of Process#signal. Read more at #9006.

We also deprecated the top-level fork, since it won’t be available in multi-threading. If this is a stopper issue for you, Process.fork is still available. But it is no longer a public API. Read more at #9136.

Platforms

For macOS users, we fixed some compatibility issues with 10.15 (Catalina) in #9296.

For BSD users, we added support for DragonFly(BSD) in #9178.

For musl users, we fixed some weird segfaults in #9238 and fixed some empty backtraces #9267.

For Windows users, well, lots of stuff. To see the ongoing efforts don’t miss the wiki page and #5430.

Regarding actual changes in this release for Windows we have:

File improvements #9015, #9038, #9037, #9257.
IO behaviour was aligned in #9207.
Process was implemented #9047, #9021, #9122, #9112, #9149, #9310
crystal spec command deals with Windows paths in #9234
And the compiler is able to bootstrap itself on windows. #9054, #9062, #9095, #9106, #9307.

Again, note that there are still some parts of the std-lib that are not ready to work on Windows.

Tools

The main story in the doc generator tool is the support to show a version picker. An external .json file will let you specify the current and past releases to populate the version picker. Read more at #8792, #9074, and #9254.

Next steps

Please update your Crystal and report any issues. We will keep moving forward and start the development focusing on 1.0.0. We expect to release some 1.0.0-preX to iterate on some final fixes.

We acknowledge that there were a lot of cleanups in the last couple of releases. We did our best to stay below a discomfort threshold.

All the deprecated definitions will be removed for 1.0. We want a clean 1.0.

We have been able to do all of this thanks to the continued support of 84codes, Nikola Motor Company and every other sponsor. It is extremely important for us to sustain the support through donations, so that we can maintain this development pace. OpenCollective and Bountysource are two available channels for that. Reach out to crystal@manas.tech if you’d like to become a direct sponsor or find other ways to support Crystal. We thank you in advance!

Comment archive for this post