๋ณธ๋ฌธ ๋ฐ”๋กœ๊ฐ€๊ธฐ
728x90
๋ฐ˜์‘ํ˜•

Deep Learning/[์ฝ”๋“œ ๊ตฌํ˜„] DL Architecture ๊ตฌํ˜„3

[Transformer] train.py, dataset.py, config.py, Mask ๊ตฌํ˜„ํ•˜๊ธฐ - 2 (Pytorch) ์ง€๋‚œ ์‹œ๊ฐ„์— ์ด์–ด, ์˜ค๋Š˜์€ ๋‚˜๋จธ์ง€ train.py, config.py, dataset.py ํŒŒ์ผ์„ ๊ตฌํ˜„ํ–ˆ๋‹ค. https://www.youtube.com/watch?v=ISNdQcPhsts ์ด ๋ถ„ ์ฝ”๋“œ๋ฅผ ๋ฐ”ํƒ•์œผ๋กœ ๊ตฌํ˜„ํ•˜์˜€์Šต๋‹ˆ๋‹ค. 1. Dataset.py ๊ตฌํ˜„ 1-1. Bilingual Dataset ์‚ฌ์šฉํ•œ ๋ฐ์ดํ„ฐ์…‹์€ Hugging Face์—์„œ ์ œ๊ณตํ•˜๋Š” opus_books Dataset์„ ํ™œ์šฉํ•˜์˜€๋‹ค. https://huggingface.co/datasets/opus_books/viewer/en-it opus_books · Datasets at Hugging Face { "en": "Nor could I pass unnoticed the suggestion of the bleak shores of Laplan.. 2024. 2. 21.
[Transformer] ์•„ํ‚คํ…์ฒ˜ ๊ตฌํ˜„ํ•˜๊ธฐ - 1 (Pytorch) Transformer๋Š” ๋…ผ๋ฌธ์œผ๋กœ๋งŒ ์ฝ์–ด๋ดค์ง€, ์ฝ”๋“œ๋กœ ๋œฏ์–ด๋ณด๋Š” ๊ฒƒ์€ ์ฒ˜์Œ์ด๋‹ค. ๋…ผ๋ฌธ ์ €์ž๋“ค์€ ์ •๋ง ์ฒœ์žฌ๊ฐ€ ๋งž๋Š” ๊ฒƒ ๊ฐ™๋‹ค. ์œ ํŠœ๋ธŒ๋ฅผ ์ฐธ๊ณ ํ•ด์„œ ์ฝ”๋“œ๋ฅผ ๊ตฌํ˜„ํ•˜์˜€์œผ๋ฉฐ, ์ด๋ฒˆ ํฌ์ŠคํŒ…์€ ์˜ค๋กœ์ง€ ์•„ํ‚คํ…์ฒ˜์—๋งŒ ์ดˆ์ ์„ ๋งž์ท„๋‹ค. ๋ฐ์ดํ„ฐ ๋ถ€๋ถ„์€ ๋‹ค์Œ์ฃผ์— ์˜ฌ๋ฆด ์˜ˆ์ •. 1. Input Embedding ๊ตฌํ˜„ํ•˜๊ธฐ import torch import torch.nn as nn import math #Input embedding class InputEmbeddings(nn.Module): #d ์ฐจ์› ์„ค์ •, vocab size ์„ค์ •(์–ผ๋งˆ๋‚˜ ๋งŽ์€ ๋‹จ์–ด ๋„ฃ์„๊ฑด์ง€) def __init__(self,d_model : int, vocab_size : int): super().__init__() self.d_model = d_model self... 2024. 2. 17.
[UNet] copy and crop ์ฝ”๋“œ ๊ตฌํ˜„ ๋ฐ ์•„ํ‚คํ…์ฒ˜ ๊ตฌํ˜„ํ•˜๊ธฐ (Pytorch) ๋‹ค๋ฅธ ๋ธ”๋กœ๊ทธ๋“ค์—์„œ๋Š” padding์„ 2๋กœ ๋งž์ถฐ์„œ, ๋”ฐ๋กœ copy and crop์„ ํ•˜์ง„ ์•Š์•˜์ง€๋งŒ, ํ•ด๋‹น ๋…ผ๋ฌธ์˜ ๊ทธ๋ฆผ์„ ๋ณด๊ณ  ์žฌํ˜„ํ•˜๊ธฐ ์œ„ํ•ด padding=0์œผ๋กœ ์„ค์ •ํ•ด์„œ ๋ณธ ์•„ํ‚คํ…์ฒ˜๋ฅผ ์ƒˆ๋กœ ๊ตฌ์ถ•ํ•˜์˜€๋‹ค. import torch.nn as nn import torch.nn.functional as F import torch import os import numpy as np from torchvision import transforms #https://sd118687.tistory.com/8 (Mirroring extrapolate) class UNet(nn.Module): #copy and crop ํ•จ์ˆ˜ ์ •์˜ #batch_size, channels, height, width def copy_and_crop(.. 2024. 2. 8.
728x90
๋ฐ˜์‘ํ˜•

ํ‹ฐ์Šคํ† ๋ฆฌํˆด๋ฐ”