:afire: Third Man :afire: on Nostr: Multimodal models, and in particular Transformer models (thanks to their modular ...

Multimodal models, and in particular Transformer models (thanks to their modular design), can parse more modalities of data together while retaining the capabilities of their unimodal peers. Take MetaTransformer, for example: below there are listed twelve different modalities, including time series data and images. Agentic behaviour could be steered with special Reinforcement Learning training and even prompting techniques.

github.com/invictus717/MetaTransformer