#71　Understanding opaque types

Introduction

Rust is a systems programming language known for its safety and concurrency features. Among its many advanced type system capabilities are opaque types and associated types, which play crucial roles in ensuring code abstraction and flexibility. One day, I spent quite some time figuring out why the following code wouldn't compile:

fn iterator() -> impl Iterator<Item = u32> {
    vec![1, 2, 3].into_iter()
}

trait Foo {
    fn iterator(&self) -> impl Iterator<Item = u32>;
}

struct Struct;

impl Foo for Struct {
    fn iterator(&self) -> impl Iterator<Item = u32> {
        vec![4, 5, 6].into_iter()
    }
}

fn main() {
    let mut array = vec![iterator()];
    
    let x = Struct;
    array.extend(vec![x.iterator()]);
}

You can check this code at playground.

Forget about what is Iterator for now: the code seems to be fine because both iterator method returns the type of implementing Iterator. However, it is actually incorrect.
The compiler raises an error for this code.

error[E0271]: type mismatch resolving `<Vec<impl Iterator<Item = u32>> as IntoIterator>::Item == impl Iterator<Item = u32>`
   --> src/main.rs:21:18
    |
1   | fn iterator() -> impl Iterator<Item = u32> {
    |                  ------------------------- the expected opaque type
...
12  |     fn iterator(&self) -> impl Iterator<Item = u32> {
    |                           ------------------------- the found opaque type
...
21  |     array.extend(vec![x.iterator()]);
    |           ------ ^^^^^^^^^^^^^^^^^^ expected opaque type, found a different opaque type
    |           |
    |           required by a bound introduced by this call
    |
    = note: expected opaque type `impl Iterator<Item = u32>` (opaque type at <src/main.rs:1:18>)
               found opaque type `impl Iterator<Item = u32>` (opaque type at <src/main.rs:12:27>)
    = note: distinct uses of `impl Trait` result in different opaque types
note: required by a bound in `extend`
   --> /playground/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/iter/traits/collect.rs:415:31
    |
415 |     fn extend<T: IntoIterator<Item = A>>(&mut self, iter: T);
    |                               ^^^^^^^^ required by this bound in `Extend::extend`

For more information about this error, try `rustc --explain E0271`.
error: could not compile `playground` (bin "playground") due to 1 previous error

The issue arises because each iterator method returns different types. This discrepancy is why the code does not compile. In this blog, we will explore the differences between these two concepts and understand their respective use cases.

Understanding Opaque Types

Opaque types in Rust allow you to hide the implementation details of a type, providing a level of abstraction. This is useful for encapsulating functionality and ensuring that users of your type cannot rely on its internal structure.

Let's revisit the function from the introduction

fn iterator() -> impl Iterator<Item = u32> {
    vec![1, 2, 3].into_iter()
}

Here, iterator returns an impl Iterator<Item = u32>. The impl Trait syntax is used to specify that the function returns some type that implements the Iterator trait with Item = u32, without exposing the actual type. This is an example of an opaque type.

Using impl Trait in function return positions helps to hide the concrete type of the iterator, enabling you to change the underlying type without affecting code that depends on the function. However, the limitation is that each function returning an impl Trait is considered to have a distinct, unnamed type, even if the trait and the associated types are the same.

Opaque types are great for:

1. Encapsulation: Hiding implementation details and exposing only necessary interfaces.
2. Flexibility: Allowing changes to the underlying implementation without breaking dependent code.

Use Cases

Opaque types are best suited for scenarios where you want to hide complex internal details and provide a simple API. Associated types are ideal for generic programming where the type relationships need to be defined and varied based on implementations.

How to resolve the problem?

I think there are many for solution for this issue. One of solution is using dynamic dispatch. Instead of using impl Trait in the trait method return type, we can use Box type:

fn iterator() -> Box<dyn Iterator<Item = u32>> {
    Box::new(vec![1, 2, 3].into_iter())
}

trait Foo {
    fn iterator(&self) -> Box<dyn Iterator<Item = u32>>;
}

struct Struct;

impl Foo for Struct {
    fn iterator(&self) -> Box<dyn Iterator<Item = u32>> {
        Box::new(vec![4, 5, 6].into_iter())
    }
}

fn main() {
    let mut array: Vec<Box<dyn Iterator<Item = u32>>> = vec![iterator()];
    
    let x = Struct;
    array.extend(vec![x.iterator()]);

    for iter in array {
        for item in iter {
            print!("{}", item);
        }
    }
}

In this updated version, using Box<dyn Iterator<Item = u32>> enables dynamic dispatch, allowing you to store and work with different types that implement the same trait.

If we run this code, we would get like this

cargo run

# 123456

you can check here as well.

Conclusion

Understanding opaque types is probably hard for the first time. However, you know the basic what is opaque types and when to use it. In addition, I introduced the solution how to resolve the issue. Hopefully, the blog is helpful for you.
Thank you for reading.