๋ณธ๋ฌธ ๋ฐ”๋กœ๊ฐ€๊ธฐ
728x90
๋ฐ˜์‘ํ˜•

Transformer2

[Transformer] train.py, dataset.py, config.py, Mask ๊ตฌํ˜„ํ•˜๊ธฐ - 2 (Pytorch) ์ง€๋‚œ ์‹œ๊ฐ„์— ์ด์–ด, ์˜ค๋Š˜์€ ๋‚˜๋จธ์ง€ train.py, config.py, dataset.py ํŒŒ์ผ์„ ๊ตฌํ˜„ํ–ˆ๋‹ค. https://www.youtube.com/watch?v=ISNdQcPhsts ์ด ๋ถ„ ์ฝ”๋“œ๋ฅผ ๋ฐ”ํƒ•์œผ๋กœ ๊ตฌํ˜„ํ•˜์˜€์Šต๋‹ˆ๋‹ค. 1. Dataset.py ๊ตฌํ˜„ 1-1. Bilingual Dataset ์‚ฌ์šฉํ•œ ๋ฐ์ดํ„ฐ์…‹์€ Hugging Face์—์„œ ์ œ๊ณตํ•˜๋Š” opus_books Dataset์„ ํ™œ์šฉํ•˜์˜€๋‹ค. https://huggingface.co/datasets/opus_books/viewer/en-it opus_books · Datasets at Hugging Face { "en": "Nor could I pass unnoticed the suggestion of the bleak shores of Laplan.. 2024. 2. 21.
[Transformer] ์•„ํ‚คํ…์ฒ˜ ๊ตฌํ˜„ํ•˜๊ธฐ - 1 (Pytorch) Transformer๋Š” ๋…ผ๋ฌธ์œผ๋กœ๋งŒ ์ฝ์–ด๋ดค์ง€, ์ฝ”๋“œ๋กœ ๋œฏ์–ด๋ณด๋Š” ๊ฒƒ์€ ์ฒ˜์Œ์ด๋‹ค. ๋…ผ๋ฌธ ์ €์ž๋“ค์€ ์ •๋ง ์ฒœ์žฌ๊ฐ€ ๋งž๋Š” ๊ฒƒ ๊ฐ™๋‹ค. ์œ ํŠœ๋ธŒ๋ฅผ ์ฐธ๊ณ ํ•ด์„œ ์ฝ”๋“œ๋ฅผ ๊ตฌํ˜„ํ•˜์˜€์œผ๋ฉฐ, ์ด๋ฒˆ ํฌ์ŠคํŒ…์€ ์˜ค๋กœ์ง€ ์•„ํ‚คํ…์ฒ˜์—๋งŒ ์ดˆ์ ์„ ๋งž์ท„๋‹ค. ๋ฐ์ดํ„ฐ ๋ถ€๋ถ„์€ ๋‹ค์Œ์ฃผ์— ์˜ฌ๋ฆด ์˜ˆ์ •. 1. Input Embedding ๊ตฌํ˜„ํ•˜๊ธฐ import torch import torch.nn as nn import math #Input embedding class InputEmbeddings(nn.Module): #d ์ฐจ์› ์„ค์ •, vocab size ์„ค์ •(์–ผ๋งˆ๋‚˜ ๋งŽ์€ ๋‹จ์–ด ๋„ฃ์„๊ฑด์ง€) def __init__(self,d_model : int, vocab_size : int): super().__init__() self.d_model = d_model self... 2024. 2. 17.
728x90
๋ฐ˜์‘ํ˜•

ํ‹ฐ์Šคํ† ๋ฆฌํˆด๋ฐ”