iTranslated by AI

The content below is an AI-generated translation. This is an experimental feature, and may contain errors. View original article
⛳

New Features in GHC 8.10 and GHC 9.0

に公開
Haskell
tech

I am mod_poppo, and I gave a presentation titled "GHC Trends 2021" at Haskell Day 2021. Recently, I wrote an article titled:

However, I thought the new features of GHC 8.10 and GHC 9.0 were also worth introducing, so I will summarize them here.

As with the previous article, since my interests are somewhat biased, there are some areas that I haven't covered much, such as Template Haskell and IDE-related features.

New Features in GHC 8.10

GHC 8.10.1 was released in March 2020. The official release notes can be found here:

UnliftedNewtypes Extension

I wrote about unlifted data types in the GHC 9.2 introduction article, so please refer to that.

This extension allows you to create newtypes for unlifted types.

As an example, previously only Int# had the kind TYPE 'IntRep, but using this extension:

Prelude6 :m + GHC.Exts
Prelude GHC.Exts6 :set -XUnliftedNewtypes -XMagicHash
Prelude GHC.Exts6 newtype I = I Int#
Prelude GHC.Exts6 :kind I
I :: TYPE 'IntRep

In this way, you can create a "type that has kind TYPE 'IntRep but is distinct from Int#".

In conjunction with this, coerce from the Data.Coerce module has also been extended to handle unlifted types. Its type will look something like this:

coerce :: forall {k :: RuntimeRep} (a :: TYPE k) (b :: TYPE k). Coercible a b => a -> b

StandaloneKindSignatures Extension

One of the characteristic syntaxes of Haskell is that type annotations can be written on a separate line from the implementation, like this:

foo :: Int -> String -> IO ()
foo = ...

However, until now, it was not possible to write kind annotations for types on a separate line. For example, if you wanted to explicitly state that the kind of a is Type -> Type in a type definition that takes a type argument a, you had to write:

data Foo (a :: Type -> Type) = ...

With the newly introduced StandaloneKindSignatures extension, you can use an entire line for the kind annotation:

type Foo :: (Type -> Type) -> Type
data Foo a = ...

In addition to data, this feature can be used for type (family), newtype, and class.

It seems that StandaloneKindSignatures also enables type-level polymorphic recursion.

This extension is part of GHC2021, which is available in GHC 9.2 and later.

ImportQualifiedPost Extension

import qualified Data.Vector as V

can now be written as:

import Data.Vector qualified as V

This extension is part of GHC2021, which is available in GHC 9.2 and later.

Low-latency GC

GHC's GC uses a generational copying GC, which works well for programs that do not require real-time performance. However, for programs requiring real-time properties, the pause time caused by GC (the so-called "Stop the World") becomes a non-negligible issue (or so I've heard; I don't write such programs myself, so I'm quoting others).

In GHC 8.10, to mitigate the problem of GC pause times, a new GC is introduced that is:

  • Concurrent
  • Non-moving
  • Mark-and-sweep

This GC is applied to the older generations, while the traditional copying GC is still used for the younger generations.

This new hybrid GC is disabled by default and can be enabled with the --nonmoving-gc RTS option (used like ./myprogram +RTS --nonmoving-gc).

This new mark-and-sweep GC is reportedly based on papers by Ueno, Ohori, et al. from Tohoku University.

Stop special treatment for forall in kind variables

Kind variables in GHC were originally completely distinct from type variables, but they have gradually been unified. The last bit of special treatment remaining was related to forall.

First, regarding forall for type variables, there is a "forall-or-nothing" rule. This rule states that if you use forall in a type annotation, you must quantify all type variables that appear. Here is a code example:

foo :: a -> b -> a  -- OK; implicitly universally quantified
bar :: forall a b. a -> b -> a  -- OK; all variables are quantified
baz :: forall a. a -> b -> a  -- Error; b is not quantified despite the presence of forall

However, until GHC 8.8, this rule did not apply to kind variables, and you could write code like this:

{-# LANGUAGE ExplicitForAll #-}
{-# LANGUAGE PolyKinds #-}
import Data.Proxy

foo :: forall (a :: k). Proxy a -> ()  -- forall is present, but k is not quantified
foo _ = ()

GHC 8.10 abolishes this special treatment and makes binding kind variables mandatory when using forall.

-- GHC 8.10 and later
foo :: forall (a :: k). Proxy a -> ()  -- Error: k is not bound
bar :: forall k (a :: k). Proxy a -> ()  -- OK

forall k ->: visible, dependent quantifiers

In GHC 8.0 and later, kind annotations can depend on type-level variables that appeared earlier. For example, in the type definition:

{-# LANGUAGE PolyKinds #-}
data T k (a :: k)

the kind of the second type argument depends on the first type argument. So, what is the kind of this T?

If we check in GHCi:

Prelude> :set -XPolyKinds
Prelude> data T k (a :: k) = MkT
Prelude> :kind T
T :: forall k -> k -> *

a mysterious notation forall k -> appeared. This has two properties:

  • The part following -> can depend on the bound variable (same as a regular forall).
  • It must be explicitly provided as a type argument (same as a regular type argument).

Until GHC 8.8, forall k -> could only be seen in GHCi's :kind output and could not be written in source code. In GHC 8.10, you can now write forall k -> in your source code.

{-# LANGUAGE StandaloneKindSignatures #-}
{-# LANGUAGE ExplicitForAll #-}
{-# LANGUAGE PolyKinds #-}
import Data.Kind

type T :: forall k -> k -> Type
data T k a = MkT -- the kind of a is k

-Wno-deriving-defaults

The deriving-related extensions DeriveAnyClass and GeneralizedNewtypeDeriving can conflict with each other.

{-# LANGUAGE DeriveAnyClass, GeneralizedNewtypeDeriving #-}

newtype U = U Int deriving (Show, Num) -- How is Num derived?

main = print (U 0 - U 0)

In such cases, GHC issues a warning message. This behavior has been present for a while.

Up to GHC 8.8:

DerivingTest.hs:3:35: warning:
    • Both DeriveAnyClass and GeneralizedNewtypeDeriving are enabled
      Defaulting to the DeriveAnyClass strategy for instantiating Num
      Use DerivingStrategies to pick a different strategy
    • In the newtype declaration for ‘U’
  |
3 | newtype U = U Int deriving (Show, Num) -- How is Num derived?
  |                                   ^^^

GHC 8.10 and later:

DerivingTest.hs:3:35: warning: [-Wderiving-defaults]
    • Both DeriveAnyClass and GeneralizedNewtypeDeriving are enabled
      Defaulting to the DeriveAnyClass strategy for instantiating Num
      Use DerivingStrategies to pick a different strategy
    • In the newtype declaration for ‘U’
  |
3 | newtype U = U Int deriving (Show, Num) -- How is Num derived?
  |                                   ^^^

What has changed is that the warning is now named deriving-defaults, and as a result, it is now possible to suppress this warning using -Wno-deriving-defaults.

For more details regarding deriving in GHC, please refer to the following article:

GHCi automatically enables -fobject-code when UnboxedTuples or UnboxedSums are used

As the heading suggests.

bitReverse

Depending on the instruction set, there are sometimes instructions for reversing the bits of an unsigned integer (e.g., ARM). Built-in functions corresponding to these have been added.

They can be used in the Data.Word module as follows:

bitReverse8 :: Word8 -> Word8
bitReverse16 :: Word16 -> Word16
bitReverse32 :: Word32 -> Word32
bitReverse64 :: Word64 -> Word64

As of GHC 9.2, none of the NCGs (Native Code Generators) lower these to dedicated instructions; instead, they call implementations written in C. If you use the LLVM backend, they will be converted to LLVM's llvm.bitreverse.* intrinsics, so there is a possibility they will be compiled into the corresponding instructions if the target supports them.

New Features in GHC 9.0

GHC 9.0.1 was released in February 2021. The official release notes are:

LinearTypes Extension

Linear types have arrived. For detailed explanations, please refer to other articles and slides.

Under the LinearTypes extension, the syntax a %m -> b for types is added. The regular -> and %m -> are desugared as follows:

a -> b     ~~>  a %'Many -> b
a %1 -> b  ~~>  a %'One -> b
a %m -> b  ~~>  FUN m a b

FUN, Many, and One are defined as follows:

-- GHC.Exts
type FUN :: forall (n :: Multiplicity) -> forall {q :: RuntimeRep} {r :: RuntimeRep}. TYPE q -> TYPE r -> Type
data FUN m a b

-- GHC.Types (ghc-prim)
data Multiplicity = Many | One
type MultMul :: Multiplicity -> Multiplicity -> Multiplicity

Note that up until the alpha version of GHC 9.0.1, the syntax was #-> instead of %1 ->. Please be careful when reading older articles.

QualifiedDo Extension

There are things that are monad-like but don't become instances of the Monad class. These include indexed monads, graded monads, and linear versions of monads.

Previously, if you wanted to use do-notation for such "monad-like things that don't become instances of the Monad class," your only option was to use the RebindableSyntax extension. However, RebindableSyntax affects many syntactic constructs and was too heavy-handed for simply wanting to "replace do-notation."

The QualifiedDo extension allows you to choose the desugaring method for individual do-blocks.

In terms of code, you qualify do with a module name. If the module name is SugoiMonad, you write SugoiMonad.do. By doing this:

  • SugoiMonad.>>= is used for desugaring x <- u.
  • SugoiMonad.>> is used for desugaring u; ....
  • SugoiMonad.fail is used for desugaring pattern matches that can fail, like pat <- u.

If the ApplicativeDo extension is enabled, SugoiMonad.<$>, SugoiMonad.<*>, SugoiMonad.join, and SugoiMonad.return will also be used. If the RecursiveDo extension is enabled, SugoiMonad.mfix and SugoiMonad.return will be used as well.

One thing to be careful about is that the >>= or return written directly in the source code will not be automatically qualified with the module name. In other words, you would write:

SugoiMonad.do
  x <- foo SugoiMonad.>>= bar
  SugoiMonad.return (baz x)

instead of:

SugoiMonad.do
  x <- foo >>= bar
  return (baz x)

LexicalNegation Extension and Changes to NegativeLiterals

In Haskell, writing (operator operand) allows you to obtain a function equivalent to \x -> x operator operand (a section)... unless the operator is the minus sign -. If the operator is minus, it is interpreted as a unary minus.

The traditional Haskell solution (the way to write a subtraction section) was to write subtract operand instead of (- operand), but LexicalNegation provides an alternative.

When the LexicalNegation extension is enabled, binary and unary operations are toggled based on the presence or absence of whitespace around the minus sign -.

x-y   -- Binary operation
x- y  -- Binary operation
x -y  -- Unary minus: x (-y)
x - y -- Binary operation

(- x) -- Section: \y -> y - x
(-x)  -- Unary minus: negate x

Also, unary minus binds more tightly than other binary operators.

In GHC 9.0, changes were also made to the NegativeLiterals extension, so that even under NegativeLiterals, x-1 is interpreted as x - 1 rather than x (-1). For detailed rules, please see the GHC proposal.

Simplified Subsumption

Previously, two functions with different forall positions, such as:

foo :: forall x y. x -> y -> Int
bar :: forall x. x -> forall y. y -> Int

were not distinguished (except in the context of type applications). For example, f bar would compile successfully against:

f :: (forall x y. x -> y -> Int) -> Int

This was because implicit η-conversion was performed:

f bar ~~> f (\x y -> bar x y)

However, from GHC 9.0 onwards, the order and position of type variables are distinguished. As a result, there will be more instances where manual η-conversion is required.

forall {a}.

I wrote in Reason why TypeApplications cannot be used for literals and its countermeasures, or points to note about TypeApplications that TypeApplications cannot always be used for polymorphic functions, and depending on how the function is defined, it might not be available.

Functions with type variables that cannot be applied are displayed with the type variables enclosed in curly braces {} when the type is shown in GHCi with :set -fprint-explicit-foralls enabled:

ghci> :set -fprint-explicit-foralls
ghci> foo x y = x + y
ghci> :type foo
foo :: forall {a}. Num a => a -> a -> a

In GHC 9.0, it becomes possible to write curly braces for the type variables in forall within the source code as well.

{-# LANGUAGE ExplicitForAll #-}

foo :: forall {a} b. a -> b -> a
foo x y = x
-- Applying foo @Int results in b=Int

{-# LANGUAGE ExplicitForAll #-}
{-# LANGUAGE GADTSyntax #-}
{-# LANGUAGE PolyKinds #-}

data MyProxy a where
  MyProxy :: forall {k} (a :: k). MyProxy a
-- Applying MyProxy @Int results in a=Int

WinIO

A new I/O manager for Windows has been introduced. In GHC 9.0, it is positioned as a preview version and is disabled by default. It can be enabled with the --io-manager=native RTS option (e.g., ./myprogram.exe +RTS --io-manager=native).

One difference between the traditional I/O manager and WinIO is the use of the Wide API for console input and output. This means Unicode characters can be read and written even if the console's code page is not 65001:

> cat hello.hs
main = putStrLn "\x1F97A"
> chcp
Current code page: 932
> ghc-8.10.7 -fforce-recomp hello.hs
[1 of 1] Compiling Main             ( hello.hs, hello.o )
Linking hello.exe ...
> .\hello.exe
hello.exe: <stdout>: commitBuffer: invalid argument (invalid character)
> ghc-9.0.1 -rtsopts -fforce-recomp hello.hs
[1 of 1] Compiling Main             ( hello.hs, hello.o )
Linking hello.exe ...
> .\hello.exe
hello.exe: <stdout>: commitBuffer: invalid argument (invalid character)
> .\hello.exe +RTS --io-manager=native
🥺

A New Implementation for Big Integers: ghc-bignum

In previous versions of GHC, the implementation of arbitrary-precision integers (Integer and Natural types) was provided by either:

  • integer-gmp: Implementation using GMP
  • integer-simple: A pure Haskell implementation

One of these would be used. However, this system had several issues:

  1. The efficiency of integer-simple was not good (representation using a linked list like [Word]).
  2. Accessing the internal representation of the Integer type from general packages was cumbersome.
    • The internal representation of the Integer type differed between integer-gmp and integer-simple (requiring switching via CPP). The packages to depend on also differed (requiring switching via package flags).
  3. It was difficult to add new backends (e.g., OpenSSL, which has a more permissive license than GMP).
-- Internal representation in integer-gmp
data Integer = S# !Int#    -- Values that fit in one word
             | Jp# !BigNat -- Positive big integer
             | Jn# !BigNat -- Negative big integer

-- Internal representation in integer-simple
data Integer = Positive !Positive
             | Negative !Positive
             | Naught

To address this, the ghc-bignum package was introduced. In ghc-bignum, the internal representations of the Integer and Natural types are unified.

-- ghc-bignum
data Integer = IS !Int#    -- Equivalent to S# in integer-gmp
             | IP !BigNat# -- Equivalent to Jp# in integer-gmp
             | IN !BigNat# -- Equivalent to Jn# in integer-gmp

The actual operation implementations are provided as either GMP-based or pure Haskell (the latter is called native). The pure Haskell implementation is expected to be faster than integer-simple.

For compatibility with packages that previously accessed the internal representation of the Integer type directly, the integer-gmp package continues to be provided (offering S#, Jp#, and Jn# using PatternSynonyms).

Removal of Support for 32-bit Windows and Windows Vista

As the heading suggests. The winapi pseudo-calling convention ended up not being implemented before 32-bit Windows support was dropped. A shame.

Note that GHC 8.6.5 seems to be the last version with an official binary release for 32-bit Windows. If you want to run a newer GHC on 32-bit Windows, you would likely need to build it yourself.

GHCi Prompt Changed

Previously, GHCi displayed the module names in scope in the prompt, such as Prelude>. In GHC 9.0, the prompt simply becomes ghci>.

If you want to see the list of modules currently in scope, please run :show imports.

If you prefer the old prompt style, you can run (or add to your .ghci file) the following:

:set prompt "%s> "
:set prompt-cont "%s| "

Length of Unboxed String Literals: cstringLength#

Regarding unboxed string literals, please refer to How Haskell string literals are compiled in GHC. In short, these are literals for null-terminated byte sequences.

In GHC 9.0, a function cstringLength# :: Addr# -> Int# has been added to the GHC.CString / GHC.Exts modules. This is equivalent to the C strlen function and returns the number of bytes up to the first NUL.

The important point is that this function is subject to constant folding by the compiler. This ensures that when constructing user-defined string types from literals, the number of bytes doesn't need to be counted at runtime.

touch# and keepAlive#

When using objects managed by Haskell's GC in C FFI, it's problematic if the GC triggers during an FFI call and releases the object. To prevent this, it was traditional practice to insert a touch function after the FFI call.

do arr <- newPinnedByteArray len
   ... use the address of arr in C FFI ...
   touch arr -- A magic spell to prevent arr from being released during FFI

However, if the FFI part results in an infinite loop or throws an exception, GHC may determine that control never reaches the touch call and delete the code (this is known as dead-continuation elimination). If this happens, the memory area used by the FFI may be reclaimed by the GC, causing serious issues.

To address this, mitigation measures like marking potentially infinite loops with NOINLINE have been taken in the past, but GHC 9.0 introduces a new primitive as a fundamental solution:

keepAlive# :: forall (q :: RuntimeRep) (a :: TYPE q) (r :: RuntimeRep) (b :: TYPE r).
              a -> State# RealWorld -> (State# RealWorld -> b) -> b

In GHC 9.0 and later, withForeignPtr uses this keepAlive#.

Note that keepAlive# does not completely replace the traditional touch# primitive; the traditional touch# primitive remains available (to be used at the programmer's responsibility).

Data.List.singleton and Data.List.NonEmpty.singleton

Functions that return a list consisting of a single element have been added. Previously, many people used patterns like (:[]).

-- Data.List
singleton :: a -> [a]

-- Data.List.NonEmpty
singleton :: a -> NonEmpty a

The discussion about adding singleton to Data.List eventually branched into topics like the monomorphization of Data.List, but that is another story.

Discussion