Janet's Shenanigans

libre

the messages view showing a file thumbnail

Tok has seen a handful of improvements this week, one being that file messages now show a thumbnail if available.

Music View

music view

Tok can now show you all songs that have been uploaded to a chat.

Hop To Message

Tok now allows you to hop to a message by tapping on it, no matter how far back in history it is.

Bugfixes:

  • messages with newlines can be sent
  • formatting works again
  • nix files updated
  • the message text field no longer keeps the last formatting applied when you send a message

Obtaining Tok

Tok can be built from source from https://invent.kde.org/network/tok.

There's a Telegram room for Tok available at https://t.me/kdetok, where you can come on and chat about anything Tok related, such as asking questions on using or building Tok.

Contributing

Interested in contributing? Come on by the dev chat and say hello!

Tags: #libre

syntax highlighting being demonstrated in tok

Tok has had many changes since the last time I made one of these blog posts, the biggest one being that code blocks are syntax highlighted!

Tok uses KSyntaxHighlighting, the same syntax highlighting engine that powers Kate, KWrite, and other KDE applications that feature syntax highlighting.

Additionally, messages containing codeblocks are able to grow horizontally in width beyond the usual message size, letting you read horizontally wide code easier.

Emoji Completion

emoji completion

Tok now displays autocompletion for :emojis:, making the process of typing in emojis much more seamless.

Edited Indicator

edited indicator

Tok now indicates when a message has been edited by the sender.

Jump To Start Buttons

jump to start

Tok now has buttons that allow you to hop back to the start of various views, such as the chats list and the messages view.

Improved In-Window Menubar

better in-window menubar, featuring visual changes

Tok's in-window menubar now has various improvements, such as using the colour of the rest of the header area, as well as the right sidebar respecting the menubar's appearance.

Proxies

proxy support

Tok now supports configuring proxies, allowing you to access Telegram in countries that don't want you to access Telegram.

And in true anticonvergent fashion, Tok has a dedicated mobile UI for proxies instead of simply using the desktop UI on mobile or the mobile UI on desktop.

proxies on mobile

Better Notifications

better notifications

Tok notifications now display more information about the message, support more message types, and display the profile picture of the chat you're receiving the notification from.

mentions

Tok now renders mentions as links that display the user's profile when clicked.

Minor UI Improvements

Chat list unread indicators now become pill-shaped whenever they grow horizontally.

The typing indicator in the chat list is now accent-coloured, like the typing indicator in the header.

Translation Fixes

Tok now correctly loads translation files, allowing it to render in non-English languages.

Optimisations

Tok's startup time has been optimised by a few hundred milliseconds.

Bugfixes

Tok no longer resets the scroll position of the chats list whenever chats are moved.

Alt-Up, Alt-Down, and Ctrl-K keyboard shortcuts work again.

Formatting is generally less buggier, and message formats better match how they're supposed to look.

Obtaining Tok

Tok can be built from source from https://invent.kde.org/network/tok.

There's a Telegram room for Tok available at https://t.me/kdetok, where you can come on and chat about anything Tok related, such as asking questions on using or building Tok.

Contributing

Interested in contributing? Come on by the dev chat and say hello!

Tags: #libre

This blog post goes over the set of tools you have to work with when doing QtQuick UIs that need to perform something asynchronously.

Data Reactivity

Probably the oldest tool in the toolbox, as it's foundational to QML's construction, and one you should already know how to use.

BooksModel {
    id: booksModel
}
Button {
    text: `The model has ${booksModel.count} items`
}
Button {
   onClicked: booksModel.refresh()
}

QML is reactive, meaning that when you change a property, dependent bindings update automatically.

When applied to asynchrony, this means that asynchronous events or requests can simply mutate an object's properties on completion, and the UI will change when it does.

This is the most common form of asynchrony in a QtQuick UI, and it should be your first goto when you need to perform something asynchronously and display results in the UI, as it plays nicely with a declarative UI.

This is generally the right pattern to use when you can describe the asynchronous process as “the UI displays the current state of a model/object, which changes in response to events.”

Thennables

The newest tool in the toolbox (at least in QML), Thennables play nicely with other cool and shiny things, such as C++20 Coroutines.

Button {
    text: "Do it"
    onClicked: getNewButtonTextFromTheInternet().then((it) => this.text = it)
}

Implementations of the Thennable pattern can be found in Qt Automotive, as the QIviPendingReply family of classes, and in Croutons, as the Future family of classes. These allow you to make an object in C++ that you can return as a Thennable to QML after scheduling a task that'll fufill or reject the thennable from the C++ side. On the backend, it usually looks like this:

QIviPendingReply<bool> foo;

// runs asynchronously
client->getStatus([foo](bool isOnline) mutable {
    foo.setSuccess(response);
});

return foo;

Or, if you're using Croutons + C++20 Coroutines...

auto response = co_await client->getStatus();

co_return response.isOnline();

This is generally the right pattern to use when you can describe the flow of asynchronous actions as an “and then” chain. For example, “make a request and then display a notification to the user if it succeeds or fails”. If the asynchronous process can take advantage of data reactivity, this is probably not the correct pattern to use.

Task Objects

This is rarely the right pattern, and people that have to use your API will probably hate you if you ship this in a library. Despite that, it's a (thankfully uncommon) thing in the wild, so I'll document it here.

FileDownload {
    id: fileDownloader

    url: "http://placehold.it/350x150"

    onStarted: status.text = "Starting download..."
    onError: status.text = "Download failed"
    onProgressChanged: status.text = `Download ${progress}% done`
    onFinished: status.text = "Download finished"
}
Button {
	text: "Download File"
	onClicked: fileDownloader.start()
}

This essentially consists of using a full-fledged object description in QML to describe a task, and to connect to signals on it. The object needs to be given an id, and that id needs to be used to invoke a .start() method or setting a running property to true.

Task objects have many disadvantages: – trying to describe an imperative process directly with a declarative language, which goes as well as you'd expect – require creating and binding properties for an extra object, which can add up fast, even if the object's task isn't actually triggered by a user action. e.g., 50 download buttons with corresponding FileDownload objects, and only a few of those buttons will ever get pressed. – non-UI matters in an otherwise UI-only language

You can often recognise task objects by distinct “input” properties and “output” signals (less commonly, “output” properties) for a given task.

Instead of using a task object, considering using a Thennable, or better utilising reactive data.

Workers

I've honestly never seen the WorkerScript type get used in an actual program, so I'm not sure what their practical application is. Despite that, these are another option for currency, so I'll describe their behaviour.

A WorkerScript is a wrapper around the JS notion of a WebWorker, which runs in its own thread and receives and sends messages from the main thread. This is the only way to get data in and out of them, so you have to structure it as messages.

Tags: #libre

mobile upload dialogue

Tok now has TWO new completely revamped upload dialogues: one for desktop, and one for mobile. Despite how much of the app is spent using stuff that isn't the upload dialogues, the upload dialogues took a LOT of time and work to get implemented correctly.

The mobile dialogue makes it convenient to browse through your most recent photos, videos, music, and files to share them with your friends.

video upload dialogue on mobile

music upload dialogue on mobile

files upload dialogue on mobile

On desktop, the upload dialogue now offers a preview of your file, and for images, the option to compress it.

photo upload on desktop

video upload on desktop

music upload on desktop

file upload on desktop

Sending State

send state

The sending state of outgoing messages is now displayed with a little icon by the timestamp.

Adjusted Chat List Look

adjusted chat list look

The look of the chat list has been slightly adjusted, to make it look more like other KDE apps. More information is shown, such as the sending status of outgoing messages and the timestamp of the latest message.

Improved Pasting

Tok now has improved pasting capabilities, able to paste from apps that put images on the clipboard directly like Firefox and Spectacle now.

Obtaining Tok

Tok can be built from source from https://invent.kde.org/network/tok.

There's a Telegram room for Tok available at https://t.me/kdetok, where you can come on and chat about anything Tok related, such as asking questions on using or building Tok.

Contributing

Interested in contributing? Come on by the dev chat and say hello!

Tags: #libre

Meta: The first thing I have to announce is, that I'm tired of trying to come up with a cool name for every blog post. I'll just use This Week In Tok from now on.

Secret (E2EE) Chats

Tok now supports secret chats, which are indicated in the chat list using a lock icon. Secret chats are Telegram's E2EE solution, and since the clients (like Tok) are open source and the algorithim well-documented, you can easily verify the implementation for yourself. Wish I could say the same for some other E2EE chats.

a secret chat

Color Scheme Persistence

Tok now remembers which colour scheme you picked between restarts.

New Desktop Settings

Tok now has a revamped settings dialogue more in line with other KDE apps on desktop.

desktop settings

New Mobile Settings

Tok now has new dedicated mobile settings that offer a touch-friendly interface.

mobile settings

Creating Chats

Tok now allows you to create chats of all kinds: one-on-one private chats, secret chats, private groups, public groups, and channels.

the dialogue

Disabling Notifications

You can now disable notifications globally in Tok quickly and easily and enable them just as easily.

disabling

Image Improvements

Messages with images are now much less jumpy than they were before.

Images also now show a blurry thumbnail while loading.

loading...

System Tray

Tok now offers the ability to minimize to the system tray on desktop systems, where you can still receive notifications without a visible Tok window and disable/enable them from the tray directly.

system tray

Obtaining Tok

Tok can be built from source from https://invent.kde.org/network/tok.

There's a Telegram room for Tok available at https://t.me/kdetok, where you can come on and chat about anything Tok related, such as asking questions on using or building Tok.

Contributing

Interested in contributing? Come on by the dev chat and say hello!

Tags: #libre

tok's in-window menu bar

The past two weeks in Tok have been relatively slow, focusing on quality of life features & more user experience improvements more than sweeping new features.

Most noticeably, Tok now has an in-window menubar on desktop if you do not have a global menu bar. This provides globally available functionality in a well-organised palette of commands.

Creating Supergroups and Channels

tok's supergroup creation dialog

Tok is now capable of creating supergroups and channels.

Editing Sent Messages

tok editing sent messages

Tok can now edit messages that have been sent.

Better Sizing

tok showing a better-sized sticker

Sizing in Tok has been adjusted to make better use of screen real estate and to look less jumpy when resizing the window. Stickers no longer grow unboundedly and remain at a nice and small size, the way they were intended to be.

tok link preview

Tok can now display a preview of links sent in chat.

Better Desktop Folders Presentation

better folders

Tok now uses a vertical sidebar of folder items instead

Chat List Reordering

Tok now keeps the chat list's order up-to-date, moving chats with new messages to the top of the list and respects you moving pinned chats on other Telegram apps, moving them accordingly without the need to restart.

Bugfixes

Removed unneeded drums.

Obtaining Tok

Tok can be built from source from https://invent.kde.org/network/tok.

There's a Telegram room for Tok available at https://t.me/kdetok, where you can come on and chat about anything Tok related, such as asking questions on using or building Tok.

Contributing

Interested in contributing? Come on by the dev chat and say hello!

Tags: #libre

global menu

Tok has seen a handful of mostly user-experience focused improvements; the most noticeable one being the presence of global menu support. Tok now supports global menu bars; which are typically at least five times faster to access than window-local menus due to Fitts' Law.

GUI Editing

gui editing

Tok now has GUI editing of messages; allowing you to use a right click menu (or the new global menu) to apply formatting to your message.

Or, if you're so inclined, there's also keyboard shortcut alternatives to apply formatting to your message.

Typing Indicator

typing indicator

Tok now indicates whether the people (or bots) on the other side are typing or uploading an image/video/etc.

Video Download Progress

tok video progress

Tok now indicates how much of a video has downloaded, and how large a downloaded video will be. There is also a progress bar.

Mobile Performance Improvements

As per usual, Tok is always striving to be lightweight. Mobile was the area of focus for this changelog's performance improvements. Opening the group information should now be instantaneous; where it took a few seconds before.

No Left Gap In Channels

no left gap

No stone goes unturned in making Tok as visually polished as possible; even complicating the code in order to remove a few pixels of pointless blank space.

Tok now doesn't display a left gap on messages in channels.

Bugfixes

Removed unneeded drums.

Obtaining Tok

Tok can be built from source from https://invent.kde.org/network/tok.

The support/development room is at https://t.me/kdetok.

Contributing

Interested in contributing? Come on by the dev chat and say hello.

Tags: #libre

If you haven't read my previous blogposts on C++ coroutines, I would recommend doing so.

This post isn't an explanation of implementing something, as much as it is an explanation of something cool you can do in C++ using coroutines. Warning: this is very esoteric from a C++ perspective, and you're probably going to have problems understanding this post without knowing what algebraic effects are from other languages (mostly research ones), despite my best attempts to explain them.

In short, you can implement a rudimentary version of algebraic effects using C++ coroutines.

But what is an algebraic effect?

The simplest way to describe them is “resumable exceptions that can return a value”.

Koka explains it well with an example:

effect ask<a> {
  control ask() : a
}

This defines a generic effect ask<T> with an operation ask that returns a value of type T.

If a function wants to use this effect, it has to declare that it does, much in the same way that some languages require you to declare that your function can throw an exception.

fun ask-and-add-twice() : ask<int> int {
    return ask() + ask()
}

When you call this function, the Koka compiler checks that you handle ask somewhere in enclosing scopes, the same way that some languages check that your exceptions are being caught in a try/catch block.

Defining a handler for an effect in Koka looks like this:

fun ask-random() : random int {
  with control ask(){ resume(random-int()) }
  add-twice()
}

fun ask-const() : int {
  with control ask(){ resume(21) }
  add-twice()
}

So what does this have to do w/ C++ and coroutines?

C++ coroutines allow you to pause a function, and resume it, optionally passing in a value. This is the same thing that algebraic effects do: pause a function, and resume it, optionally passing in a value.

As a result, you can implement much of the functionality of algebraic effects in C++ using coroutines.

This is an example snippet of Crouton's rudimentary implementation of algebraic effects:

using Log = EffectVoidFun<std::function<void(QString)>>;

Effect<void> contextDependentLogging() {
    perform Log("hi");

    co_return;
}

void log() {
    {
        auto handler = Log::handler([](const QString& it) -> int {
            qDebug() << it;
        });

        contextDependentLogging();
    }
    {
        auto handler = Log::handler([](const QString& it) -> int {
            qWarning() << it;
        });

        contextDependentLogging();
    }
}

Algebraic effects are powerful in that they let your functions generic over behaviour, much in the same way that templates allow functions to be generic over types.

For example, say you're authoring a Matrix client. You can define an effect fetch that GETs/POSTs a URL. Without touching your code that uses fetch, you can transparently replace the handler for fetch in different parts of your application. Your live application could use a handler that uses the actual internet, while unit tests could use a handler that returns dummy data. Compared to #ifdef-ing different implementations, this allows different parts of an application to use different handlers at runtime, and allows library users to provide their own handlers without needing to modify the library.

There's a lot of stuff you can do with algebraic effects, and this only scratches the tip of the iceberg.

That's all for this blog post. Stay tuned for more C++-related shenanigans.

Contact Me

If you didn't understand anything here, please feel free to come to me and ask for clarification. This blog post is probably particularly confusing since it's deep in the realm of programming language geekery in the same way monads are.

Or, want to talk to me about other coroutine stuff I haven't discussed in these blog posts (or anything else you might want to talk about)?

Contact me here:

Telegram: https://t.me/pontaoski Matrix: #pontaoski:tchncs.de (prefer unencrypted DMs)

Tags: #libre

If you haven't read my previous blog post on coroutines in C++, you want to do that before reading this blog post.

In the last blog post, I explained how to construct an awaitable Future type.

If you've been tinkering with coroutines, you may have tried the following thing to allow awaiting on a pointer type:

auto operator await(QNetworkReply* it) {
    // ...
}

As far as the C++ compiler is concerned, this ain't kosher, because you're trying to define an operator for a primitive type (a pointer). Your compiler will probably tell you that this needs to be done on a class or enum type.

But that leaves the question of “how do I make a QNetworkReply* co_awaitable if I can't define co_await on a pointer type?” It is possible.

Your promise_type object has more jobs than what I covered in the last blog post. One of them is to potentially provide an await_transform function.

An await_transform function essentially “preprocesses” any values being co_awaited before the compiler attempts to look for a co_await implementation.

In code, if await_transform is defined on a promise_type, any co_await that looks like this:

auto response = co_await pendingValue;

will actually become:

auto response = co_await await_transform(pendingValue);

This is how we co_await a type you can't provide an operator co_await for: transform it into a type you can.

To integrate a QNetworkReply* into our coroutine types from before, this means we need to make an await_transform function that takes a QNetworkReply* and return a future. This is fairly simple.

auto await_transform(QNetworkReply* it) {
    auto future = Future();

    QObject::connect(it, &QNetworkReply::finished, it, [future, it]() {
        if (it->error() == QNetworkReply::NoError) {
            future.succeed(it);
        } else {
            future.fail(it);
        }
    });

    return future;
}

Make a future, mark as success if the reply succeeds, mark as failure if it doesn't succeed. Easy peasy.

Plonk this function in your promise_type from before.

This now allows you to do this:

Future fetch(QString url) {
    auto response = co_await nam.get(url);
    if (response.error() != QNetworkReply::NoError) {
        co_return response.readAll();
    }
    co_return false;
}

You can implement an await_transform for any input you like, and return anything you like, as long as you can co_await it.

The full code for this blog post can be found in a single-file format at https://invent.kde.org/-/snippets/1716. Compile with C++20 and -fcoroutines-ts -stdlib=libc++ for clang, and -fcoroutines for gcc.

A full library built on the style of coroutines introduced here (with type safe template variants) + other goodies for asynchronous Qt is available at https://invent.kde.org/cblack/croutons.

That's all for this blog post. Stay tuned for more C++-related shenanigans.

Contact Me

If you didn't understand anything here, please feel free to come to me and ask for clarification.

Or, want to talk to me about other coroutine stuff I haven't discussed in these blog posts (or anything else you might want to talk about)?

Contact me here:

Telegram: https://t.me/pontaoski Matrix: #pontaoski:tchncs.de (prefer unencrypted DMs)

Tags: #libre

Qt's networking code has always been one of its more obtuse parts, requiring using signals for something that didn't quite seem right for them. A linear flow of code would become a jumbled mess of member functions and signal connections.

When developing Challah's netcode, I quickly realised this wasn't going to suffice for the large amount of it I was going to be writing. Thus begins my journey through signal hell, arriving at many bumps before I discovered that integrating Qt's networking stuff with coroutines is possible.

Stop Zero

If you're just here to see some cool coroutine code, you can look at https://invent.kde.org/-/snippets/1711 for an single-file version of the tutorial in this blog-post, with a coroutine function already made and ready for you to play with. If you want something more full-fledged, look at https://invent.kde.org/cblack/coroutines.

But if you want to know what's actually going on behind the scenes, continue reading.

Stop One: Don't Do This

Approach one that I took to get out of signal hell was a simple while loop:

while (!reply->isFinished()) {
    QCoreApplication::processEvents();
}

This is a bad idea. Strange graphical glitches, bugs, crashes, etc. lie behind this innocuous code. Don't do it. Forcibly spinning the event loop in a lot of places causes a lot of bugs.

Step Two: Callback Hell

So, what do you do if you want to write somewhat linear code without doing that? Callbacks.

QObject::connect(val, &QNetworkReply::finished, [val, callback]() {
    if (val->error() != QNetworkReply::NoError) {
        val->deleteLater();
        callback({QStringLiteral("network failure(%1): %2").arg(val->error()).arg(val->errorString())});
        return;
    }
    
    auto response = val->readAll();
    
    protocol::chat::v1::CreateGuildResponse ret;
    if (!ret.ParseFromArray(response.constData(), response.length())) {
        val->deleteLater();
        callback({QStringLiteral("error parsing response into protobuf")});
        return;
    }
    
    val->deleteLater();
    callback({ret});
    return;
});

Every time I had to call into the netcode, I would have to provide a std::function<void(std::variant<Result,Error>)>. It's passable for a single call, but chaining them quickly violates “happy path sticks to the left”, making you wonder if you're actually writing Python with how far indented your code is.

c->call([c](auto response) {
  c->call([c](auto response) {
    c->call([c](auto response) {
      c->call([c](auto response) {
        c->call([c](auto response) {
        });
      });
    });
  });
});

Not good.

Step Three: Out of the Inferno, Into the Frying Pan

await/async as a mechanic in languages generally reduces the amount of callback hell you face by sugaring it for you. For example, in JS, this:

return fetch().then((it) => {
    return it.text
})

is equivalent to this:

return await fetch().text

Both of these return the exact same thing to the caller, but one is much easier to deal with when chained.

For a while, C++ didn't have this. Of course, with C++20, coroutines with co_await & company are now available* in most compilers.

Diversion: That Asterisk

Coroutines are technically available in most compilers, but you're going to have to do a lot of compiler-specific dances in both your code and the build system in order to pass the correct flags to enable coroutines.

I use this snippet to handle clang & gcc:

#if defined(__clang__)

#include <experimental/coroutine>
namespace ns = std::experimental;

#elif defined(__GNUC__)

#include <coroutine>
namespace ns = std;

#endif

where ns is aliased to the namespace containing coroutine-related stuff. Also note that clang will probably fail to compile stuff due to experimental/coroutine being part of libc++ while the rest of your system probably uses gcc's libstdc++. So, for all intents and purposes, gcc is your only option with coroutines on Linux.

Anyways...

Unhelpful Documentation

When hacking on coroutines, I quickly realised: the available documentation on what they were was both young and immature, as well as unhelpful in what they did have. So, this blog post is going to document them.

Parts of the Coroutine

Coroutines are largely split into two parts: the “promise” and the “awaitable”.

The promise is responsible for handling the “internal” side of any coroutine: it's what provides the return value, its yield behaviour, handling unhandled exceptions, the “await transformer”, and some other things. All coroutines need to return a type associated with one of these.

The awaitable handles the “external” side of the coroutine being coawaited. One of its first tasks is checking with the outside world whether or not the coroutine needs to suspend. If something is already available, there's no need to wait on it to become available. If the coroutine does need to wait, the awaitable receives a handle to reactivate it and is responsible for doing so, e.g. when an asynchronous network request completes and its results are available. The awaitable is also responsible for providing the final value returned to the user from coawaiting a coroutine.

As far as stuff that's directly interacting with coroutines goes, that's mostly it. However, there's still one type you need for an end-user API: the type returned by a coroutine, which is what the compiler uses to match your coroutine to a promise type.

SomeType a_coroutine() {
    auto foo = co_await another_thing();
    // ...
} 

This SomeType is what I will be calling the “future” or the “task” type, as it's basically the QFuture of the QPromise (or QFutureInterface to speak in Qt5 parlance).

C++ coroutines are very much “bring your own runtime and types”, and thankfully, that's what Qt already has, making it perfect for integrating into coroutines.

A QML-friendly future

C++ coroutines are the perfect fix to a long-standing issue in QML: representing asynchronous results in the UI in a manner that being an object property or model data can't.

So, let's get to constructing the base of our coroutines, and a type that will be suitable for passing into QML.

For ease of use, we'll want this done as a copyable Q_GADGET type with implicitly shared data. This will help us later down the road when we're passing the future type through lambdas.

(For the sake of simplicity of teaching how coroutines work, I'll be doing a simple non-template future type that talks in QVariants and doesn't have success/failure states. If you want to see code with more type safety and success/failure states, you can check out https://invent.kde.org/cblack/coroutines.)

We'll start by defining our shared data & behaviour.

class Future
{
    Q_GADGET

    struct Shared {
        QVariant result;
        std::function<void(QVariant)> then = [](QVariant) {};
        bool done = false;
    };
    QSharedPointer<Shared> d;

public:

    Future() {
        d.reset(new Shared);
    }
    Future(const Future& other) {
        this->d = other.d;
    }
};

This gives us a Future that is effectively a blob. First, let's define getters for done and result, so the outside world can actually tell what the current state of the Future is:

bool settled() const {
    return d->done;
}
QVariant result() const {
    return d->result;
}

Now we need a way to say that the future is completed and a result is available.

void succeed(const QVariant& value) const {
    if (d->done) {
        return;
    }
    d->done = true;
    d->result = value;
    d->then(d->result);
}

We don't want to trigger the callback again if we've already marked the future as done, so we abort early if we're already done.

You may notice that this function, despite being a setter semantically, is marked as const. This is mostly for ease of dealing with in lambdas later on.

Now, we need a way to register the callback function in the C++ side of things.

void then(std::function<void(QVariant)> then) const {
    d->then = then;

    if (d->done) {
        then(result());
    }
}

If the future is already done and you register a callback, you want to call it immediately. This is to handle situations like this:

Future fetch() {
  if (cache.has_thing()) {
    Future future;
    future.succeed(cache.get_thing();
    return future;
  }
  // ...
}
void main() {
    fetch().then([](QVariant) {
    });
}

If we didn't invoke the callback in then(), the callback would never be triggered as the future was returned to the caller in an already succeeded status.

Since I promised a QML-friendly future type, we should implement that now.

Q_INVOKABLE void then(const QJSValue& it) {
    then([va = it](QVariant result) mutable {
        va.call({va.engine()->toScriptValue(result)});
    });
}

We add a Q_INVOKABLE overload that takes a QJSValue, as JS functions are represented as QJSValues in C++. We then have a lambda which we pass back into the other then function, taking the QVariant, transforming it into a QJSValue, and calling the JavaScript callback with the variant.

JS usage looks like this:

future.then((it) => {
    console.warn(it)
})

If you've been following along, you should now have this class:

class Future
{

    Q_GADGET

    struct Shared {
        QVariant result;
        std::function<void(QVariant)> then = [](QVariant) {};
        bool done = false;
    };
    QSharedPointer<Shared> d;

public:

    Future() {
        d.reset(new Shared);
    }
    Future(const FutureBase& other) {
        this->d = other.d;
    }

    bool settled() const {
        return d->done;
    }
    QVariant result() const {
        return d->result;
    }

    void succeed(const QVariant& value) const {
        if (d->done) {
            return;
        }
        d->done = true;
        d->result = value;
        d->then(d->result);
    }

    void then(std::function<void(QVariant)> then) const {
        d->then = then;

        if (d->done) {
            then(result());
        }
    }
    Q_INVOKABLE void then(const QJSValue& it) {
        then([va = it](QVariant result) mutable {
            va.call({va.engine()->toScriptValue(result)});
        });
    }
};

The Backing Promise

Of course, the above class doesn't make a coroutine. For this, we need to implement the promise type.

For placement options, you have two places where you can put this: inside the future type itself, or outside of the type in an explicit coroutine_traits template specialisation.

The former would look like this:

class Future {
   // ...
   struct promise_type {
   };
};

While the latter would look like this:

template<typename ...Args>
struct ns::coroutine_traits<Future, Args...> {
    struct promise_type {
    };
};

(where ns is the namespace with the coroutine types, std:: on gcc or std::experimental on clang.)

The latter allows you to implement a promise type for any type, not just one that you have control of the declaration of.

For this blog post, I'll be going with the latter approach.

struct promise_type {
};

One of the things we'll need in our promise type is a member variable to hold the future type.

struct promise_type {
    Future _future;
};

Of course, the compiler doesn't know about this member variable, so we need to start implementing the promise_type interface:

struct promise_type {
    Future _future;
    Future get_return_object() noexcept {
        return _future;
    }
};

T get_return_object() noexcept is the exact type signature that needs to be implemented. We use noexcept here, as exceptions can cause a double free, and therefore, a crash.

Now, we need to implement two things: initial_suspend() and final_suspend(). These two functions are called at the start and end of your coroutine, and have to return a co_awaitable type.

In an expanded form, initialsuspend and finalsuspend look like this:

Future your_coroutine() {
    co_await promise.initial_suspend();
    // coroutine body here...
    co_await promise.final_suspend();
}

You could theoretically return anything you want here, but you'll likely be sticking with ns::suspend_never, which is an awaitable value that resumes execution of the coroutine immediately when co_awaited, and what we'll be using for this blog post.

Add the implementation to your promise type:

ns::suspend_never initial_suspend() const noexcept { return {}; }
ns::suspend_never final_suspend() const noexcept { return {}; }

Your promise is responsible for taking care of any values that are co_returned with the return_value function.

I recommend having two overloads: const T& (copy value) and T&& (move value).

For our promise_type, the implementation of those will simply look like this:

void return_value(const QVariant& value) noexcept
{
    _future.succeed(value);
}
void return_value(QVariant &&value) noexcept
{
    _future.succeed(std::move(value));
}

If your coroutine was returning something like a std::unique_ptr<T>, you would need the std::move in order to properly handle it, and thus why I say you should have this overload in your promise type.

With getreturnobject, initialsuspend, finalsuspend, and return_value, you only have one remaining function left to implement in your coroutine. It's quite simple.

void unhandled_exception() noexcept {

}

This is what gets called in the catch block of a try/catch block catching exceptions that the coroutine might throw. Use std::current_exception() to access the exception.

For this blog post, we'll simply be doing Q_ASSERT("unhandled exception"); in our implementation.

With that, the promise type is complete. It should look something like this:

struct promise_type {
    Future _future;

    Future get_return_object() noexcept {
        return _future;
    }

    ns::suspend_never initial_suspend() const noexcept { return {}; }
    ns::suspend_never final_suspend() const noexcept { return {}; }

    void return_value(const QVariant& value) noexcept
    {
        _future.succeed(value);
    }
    void return_value(QVariant &&value) noexcept
    {
        _future.succeed(std::move(value));
    }

    void unhandled_exception() noexcept {
        Q_ASSERT("unhandled exception");
    }
};

Remember that this needs to be within the Future type itself, or within the appropriate coroutine_traits overload.

The Awaitable

With the Future and promise completed, we don't have much left to do. We now need to implement operator co_await(Future) in order to allow the Future to be awaited.

Technically, we don't need the promise type to make a Future awaitable, only the Future and the co_await overload. Implementing a promise type allows your coroutine that's co_awaiting a Future to return another Future.

There's not really much to the co_await overload, so I'll just paste an implementation verbatim:

auto operator co_await(Future it) noexcept
{
    struct Awaiter {
        Future future;

        bool await_ready() const noexcept {
            return future.settled();
        }
        void await_suspend(ns::coroutine_handle<> cont) const {
            future.then([cont](QVariant) mutable {
                cont();
            });
        }
        QVariant await_resume() {
            return future.result();
        }
    };

    return Awaiter{ it };
}

co_await needs to return a type with the following methods:

bool await_ready() const noexcept: This function is called before suspending the coroutine to wait on the awaitable to resolve. If the function returns true, the coroutine continues execution without suspending. If the function returns false, it suspends and waits on the value.

Think back to the example of a function returning an already succeeded future to see why this is useful. If the future already holds a value, there's no need to waste time suspending the coroutine.

void await_suspend(ns::coroutine_handle<> callback) const: This function is called when the coroutine suspends with a coroutine handle passed in. For all intents and purposes, you can treat it as if it were a std::function holding a callback, except operator () resumes the coroutine instead of calling a function.

You call the coroutine_handle to resume the coroutine when whenever it's waiting on is ready, e.g. when the future in this example is marked as succeeded. You can connect this to just about anything, e.g. a QTimer::singleShot, a signal, using it as a callback, etc.

T await_resume() const: This is the value that gets assigned to the result of co_awaiting the expression:

auto it = co_await fetchFromNetwork();
  // |
  // ^ this is the value from await_resume()

Annd with all three types implemented, you now have a QML-friendly future, a promise, and an awaitable.

A Brand New Washing Machine

With your new coroutine, you can do just about anything asynchronously. This is an example function that returns a future that will be fufilled in N miliseconds:

Future timer(int duration) {
    Future it;

    QTimer::singleShot(duration, [it]() {
        it.succeed({});
    });

    return it;
}

Note that while it returns a Future, it is not a coroutine. It does not co_await. This is a completely normal function that you can call however you desire.

Making a coroutine using this function is easy:

Future thisIsACoroutine() {
    co_await timer(2000);

    co_return {};
}

Note that this function could not return void. The return type of a coroutine has to have an associated promise type, which void does not.

In a class exposed to QML...

class Singleton : public QObject
{
    Q_OBJECT

public:
    Q_INVOKABLE Future foo() {
        co_await timer(2000);

        co_return "i was returned by a coroutine";
    };
};

This can be used like so:

Button {
    anchors.centerIn: parent
    text: "a coroutine!"
    onClicked: Singleton.foo().then((val) => {
        console.warn(val)
    })
}

This will output the following in your console 2 seconds after every time you press this button:

qml: "i was returned by a coroutine"

The inline code from this blog post can be found in a single-file format at https://invent.kde.org/-/snippets/1711. Compile with C++20 and -fcoroutines-ts -stdlib=libc++ for clang, and -fcoroutines for gcc.

For a basic future+promise+awaitable deal, this is about all you need to know. However, the iceberg gets way deeper than that.

Stay tuned for more blog posts about the eldritch arcana you can pull off with coroutines. Next one will probably be about wrapping already-existing types such as a QNetworkReply* in order to make them co_await-able.

You can see more advanced (and arguably useful) code at https://invent.kde.org/cblack/coroutines.

Contact Me

Note that I wrote this blog post at 2 in the morning. If there's anything that doesn't make sense, please feel free to come to me and ask for clarification.

Or, want to talk to me about other coroutine stuff I haven't discussed in this blog post (or anything else you might want to talk about)?

Contact me here:

Telegram: https://t.me/pontaoski Matrix: #pontaoski:tchncs.de (prefer unencrypted DMs)

Tags: #libre