quickcode (version 1.0.6)

multihead_att: Multi-Head Attention

Description

This function implements a simplified version of multi-head attention for sequence data.

Usage

multihead_att(query, key, value, num_heads)

Value

A numeric matrix representing the attention output.

Arguments

query

A numeric matrix representing the query.

key

A numeric matrix representing the key.

value

A numeric matrix representing the value.

num_heads

An integer specifying the number of attention heads.

Examples

query <- matrix(rnorm(12), nrow = 3)
key <- matrix(rnorm(12), nrow = 3)
value <- matrix(rnorm(12), nrow = 3)
multihead_att(query, key, value, num_heads = 2)