Struct RotaryEncodingConfig

pub struct RotaryEncodingConfig {
    pub max_sequence_length: usize,
    pub d_model: usize,
    pub theta: f32,
}

Configuration to create a RotaryEncoding layer using the init function.

Fields

max_sequence_length: usize

Maximum sequence length of input

d_model: usize

Size of the input embedding or hidden dimension

theta: f32

Scaling factor for frequency computation. Defaults to 10000.0

Implementations

impl RotaryEncodingConfig

pub fn new(max_sequence_length: usize, d_model: usize) -> RotaryEncodingConfig

Create a new instance of the config.

impl RotaryEncodingConfig

pub fn with_theta(self, theta: f32) -> RotaryEncodingConfig

Scaling factor for frequency computation. Defaults to 10000.0

impl RotaryEncodingConfig

pub fn init<B>(&self, device: &<B as Backend>::Device) -> RotaryEncoding<B>

where B: Backend,

Initialize a new RotaryEncoding module.

Panics

Panics if the size of input embedding dimension is not even. Panics if the theta parameter is not positive.

pub fn init_with_frequency_scaling<B>( &self, scaling: impl Fn(Tensor<B, 1>) -> Tensor<B, 1>, device: &<B as Backend>::Device, ) -> RotaryEncoding<B>

where B: Backend,

Initialize a new RotaryEncoding module with a custom frequency scaling function. This is useful to apply different RoPE extensions.

Panics

Panics if the size of input embedding dimension is not even. Panics if the theta parameter is not positive.

Trait Implementations

impl Clone for RotaryEncodingConfig

fn clone(&self) -> RotaryEncodingConfig

Returns a copy of the value. Read more

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more

impl Config for RotaryEncodingConfig

fn save<P>(&self, file: P) -> Result<(), Error>

where P: AsRef<Path>,

Saves the configuration to a file.

fn load<P>(file: P) -> Result<Self, ConfigError>

where P: AsRef<Path>,

Loads the configuration from a file.

fn load_binary(data: &[u8]) -> Result<Self, ConfigError>

Loads the configuration from a binary buffer.

impl Debug for RotaryEncodingConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter. Read more

impl<'de> Deserialize<'de> for RotaryEncodingConfig

fn deserialize<D>( deserializer: D, ) -> Result<RotaryEncodingConfig, <D as Deserializer<'de>>::Error>

where D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Display for RotaryEncodingConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter. Read more

impl Serialize for RotaryEncodingConfig

fn serialize<S>( &self, serializer: S, ) -> Result<<S as Serializer>::Ok, <S as Serializer>::Error>

where S: Serializer,

Serialize this value into the given Serde serializer.