Questions tagged [encoder-decoder]
5 questions
1
vote
0 answers
Is there a term for encoder-decoder models with 0 layer encoder?
What do we call an encoder-decoder with 0 encoder layers and the cross-attention of decoder layers are directed to the outputs of the encoder embedding layer?
0-N Enc-decoder
Decoder-only with cross-attention
Prefix-LM
Decoder-only
Others
alvas
- 2,510
- 7
- 28
- 40
1
vote
0 answers
What's an encoder-decoder model that's known to do well for multilingual tasks?
In the age of decoder-only LLMs, I'll like to ask if there's any
competitive encoder-decoder architectures that are known to scale well for
multilingual seq2seq…
alvas
- 2,510
- 7
- 28
- 40
0
votes
0 answers
Predicting pregnancy codes with transformer
Im trying to predict pregnancy codes with a basic transformer model architecture. These pregnancy codes are like following prg001, prg002 to prg030. Prg001 would be antenatal screening and prg030 would be maternal outcome of delivery.
The source is…
NatalieL
- 101
- 1
0
votes
0 answers
How to get meaningful results from EncoderDecoder network for time series forecasting
I'm trying to traing an EncoderDecoder network for a multivariate time series input and a univariate time series output. In particular my dataset is composed of inputs of 32 features x 600 seconds and should produce 1 output x 300 seconds.
The MSE…
SimoV8
- 101
- 1
0
votes
0 answers
Poor performance of attention-based encoder-decoder architecture for slot filling
I’m currently doing some research on methods that tackle the intent classification and slot filling problems in NLP. One of the approaches with which I choose to start experimenting is proposed in the following…
MinhTu
- 1