PyTorch

PyTorch๋กœ ๋”ฅ๋Ÿฌ๋‹ ๋ชจ๋ธ์„ ๋งŒ๋“ค๊ณ  ํ•™์Šตํ•˜๋Š” ๋ฐฉ๋ฒ•.

PyTorch๋กœ ๋”ฅ๋Ÿฌ๋‹ ๋ชจ๋ธ์„ ๋งŒ๋“ค๊ณ  ํ•™์Šตํ•˜๋Š” ๋ฐฉ๋ฒ•.

์ƒ์œ„ ๊ฐœ๋…: ๋”ฅ๋Ÿฌ๋‹

11 posts · ์•„์นด์ด๋ธŒ์—์„œ ํ•„ํ„ฐ๋กœ ๋ณด๊ธฐ

(์„ค๋ช…์ถ”๊ฐ€) ์›จ์ดํŠธ ์ดˆ๊ธฐํ™” (Weight Initialization)

1. ์›จ์ดํŠธ ์ดˆ๊ธฐํ™”๋ž€? ๋”ฅ๋Ÿฌ๋‹์—์„œ ์›จ์ดํŠธ ์ดˆ๊ธฐํ™”(Weight Initialization)๋Š” ์‹ ๊ฒฝ๋ง์˜ ๊ฐ€์ค‘์น˜๋ฅผ ํ•™์Šต ์ „์— ์„ค์ •ํ•˜๋Š” ๊ณผ์ •์ž…๋‹ˆ๋‹ค. ์ดˆ๊ธฐํ™” ๋ฐฉ์‹์— ๋”ฐ๋ผ ๋ชจ๋ธ์˜ ํ•™์Šต ์†๋„, ์„ฑ๋Šฅ, ์•ˆ์ •์„ฑ์ด ํฌ๊ฒŒ ๋‹ฌ๋ผ์งˆ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ์ ์ ˆํ•œ ์ดˆ๊ธฐํ™” ๋ฐฉ์‹์€ ํ›ˆ๋ จ์„ ๊ฐ€์†ํ™”ํ•˜๊ณ , ์ตœ์ ํ™” ๊ณผ์ •์—์„œ ์•ˆ์ •์ ์ธ ํ•™์Šต์„ ๋ณด์žฅํ•˜๋ฉฐ, ๊ทธ๋ž˜๋””์–ธํŠธ ์†Œ์‹ค ๋ฐ ํญ๋ฐœ ๋ฌธ์ œ๋ฅผ ๋ฐฉ์ง€...

[Graph] 4์žฅ. Graph Neural Networks: Algorithms

1. Introduction ๊ทธ๋ž˜ํ”„ ๊ตฌ์กฐ ๋ฐ์ดํ„ฐ๋Š” ๋ณต์žกํ•œ ๊ด€๊ณ„์™€ ์ƒํ˜ธ์ž‘์šฉ์„ ๋ชจ๋ธ๋งํ•˜๋Š” ๋ฐ ๋งค์šฐ ์œ ์šฉํ•ฉ๋‹ˆ๋‹ค. ์ด๋Ÿฌํ•œ ๋ฐ์ดํ„ฐ๋ฅผ ํšจ๊ณผ์ ์œผ๋กœ ๋ถ„์„ํ•˜๊ณ  ํ•™์Šตํ•˜๊ธฐ ์œ„ํ•ด ๊ทธ๋ž˜ํ”„ ์‹ ๊ฒฝ๋ง(Graph Neural Networks, GNN)๊ณผ ๊ทธ๋ž˜ํ”„ ์ž„๋ฒ ๋”ฉ(Graph Embedding) ๊ธฐ๋ฒ•์ด ๊ฐœ๋ฐœ๋˜์—ˆ์Šต๋‹ˆ๋‹ค. 1.1 ๊ทธ๋ž˜ํ”„ ์‹ ๊ฒฝ๋ง ๋ชจ๋ธ vs ๊ทธ๋ž˜ํ”„ ์ž„๋ฒ ๋”ฉ ๊ทธ๋ž˜...

[๊ฐœ๋…] Deep Learning Normalization Techniques

๋”ฅ๋Ÿฌ๋‹์—์„œ์˜ ์ •๊ทœํ™” ๊ธฐ๋ฒ• ์ถœ์ฒ˜: https://theaisummer.com/normalization/ ์ •๊ทœํ™”์˜ ์ •์˜์™€ ๋ชฉ์  ์ •๊ทœํ™”(Normalization)๋Š” ๋ฐ์ดํ„ฐ์˜ ์Šค์ผ€์ผ์„ ์กฐ์ •ํ•˜๋Š” ๊ณผ์ •์œผ๋กœ, ๋จธ์‹ ๋Ÿฌ๋‹๊ณผ ๋”ฅ๋Ÿฌ๋‹์—์„œ ๋ชจ๋‘ ์ค‘์š”ํ•œ ์—ญํ• ์„ ํ•ฉ๋‹ˆ๋‹ค. ๊ทธ๋Ÿฌ๋‚˜ ์ „ํ†ต์ ์ธ ๋จธ์‹ ๋Ÿฌ๋‹์—์„œ์˜ ์ •๊ทœํ™”์™€ ๋”ฅ๋Ÿฌ๋‹์—์„œ์˜ ์ •๊ทœํ™”๋Š” ๊ทธ ๋ชฉ์ ๊ณผ ๋ฐฉ๋ฒ•์— ์žˆ์–ด ์•ฝ๊ฐ„์˜ ์ฐจ์ด๊ฐ€ ...

[ํŒŒ์ดํ† ์น˜] ํŒŒ์ดํ† ์น˜๋กœ CNN ๋ชจ๋ธ์„ ๊ตฌํ˜„ํ•ด๋ณด์ž! (ResNetํŽธ)

์•ˆ๋…•ํ•˜์„ธ์š”! ์ง€๋‚œ๋ฒˆ ํฌ์ŠคํŠธ์ธ VGGNet๊ณผ GoogleNet ์ดํ›„๋กœ ์˜ค๋Š˜์€ ResNet ๊ด€๋ จ ํฌ์ŠคํŠธ์ž…๋‹ˆ๋‹ค. 2๋ฒˆ์— ๊ฑธ์นœ ํฌ์ŠคํŒ…์—์„œ ์†Œ๊ฐœ๋“œ๋ ธ๋‹ค์‹œํ”ผ ์ปดํ“จํ„ฐ ๋น„์ „ ๋Œ€ํšŒ ์ค‘์— ILSVRC (Imagenet Large Scale Visual Recognition Challenges)์ด๋ผ๋Š” ๋Œ€ํšŒ๊ฐ€ ์žˆ๋Š”๋ฐ, ๋ณธ ๋Œ€ํšŒ๋Š” ๊ฑฐ๋Œ€ ์ด๋ฏธ์ง€๋ฅผ 1000๊ฐœ์˜ ์„œ๋ธŒ์ด๋ฏธ์ง€๋กœ ๋ถ„๋ฅ˜...

[ํŒŒ์ดํ† ์น˜] ํŒŒ์ดํ† ์น˜๋กœ CNN ๋ชจ๋ธ์„ ๊ตฌํ˜„ํ•ด๋ณด์ž! (GoogleNetํŽธ)

์•ˆ๋…•ํ•˜์„ธ์š”! ์ง€๋‚œ๋ฒˆ ํฌ์ŠคํŠธ์ธ VGGNet ์ดํ›„๋กœ ์˜ค๋Š˜์€ GoogleNet ๊ด€๋ จ ํฌ์ŠคํŠธ์ž…๋‹ˆ๋‹ค. ๋‹ค์Œ ํฌ์ŠคํŠธ๋Š” ResNet์œผ๋กœ ์ฐพ์•„๋ต™๋„๋ก ํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค. ์ง€๋‚œ๋ฒˆ์—๋„ ์†Œ๊ฐœ๋“œ๋ ธ๋‹ค์‹œํ”ผ ์ปดํ“จํ„ฐ ๋น„์ „ ๋Œ€ํšŒ ์ค‘์— ILSVRC (Imagenet Large Scale Visual Recognition Challenges)์ด๋ผ๋Š” ๋Œ€ํšŒ๊ฐ€ ์žˆ๋Š”๋ฐ, ๋ณธ ๋Œ€ํšŒ๋Š” ๊ฑฐ๋Œ€ ์ด๋ฏธ์ง€๋ฅผ 1...

[ํŒŒ์ดํ† ์น˜] ํŒŒ์ดํ† ์น˜๋กœ CNN ๋ชจ๋ธ์„ ๊ตฌํ˜„ํ•ด๋ณด์ž! (VGGNetํŽธ)

์•ˆ๋…•ํ•˜์„ธ์š”! ์˜ค๋Š˜ ํฌ์ŠคํŒ…๋ถ€ํ„ฐ ๋‹ค์Œ๋‹ค์Œ ํฌ์ŠคํŒ…๊นŒ์ง€๋Š” CNN ๋ชจ๋ธ์˜ ๋ผˆ๋Œ€๊ฐ€ ๋˜๋Š” ๋ชจ๋ธ๋“ค์ธ VGGNet, GoogleNet, ResNet์„ ์†Œ๊ฐœํ•˜๊ณ  ์ด๋ฅผ ๊ตฌํ˜„ํ•ด๋ณด๋Š” ์‹œ๊ฐ„์„ ๊ฐ–๋„๋ก ํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค! :) ์ด๋ฒˆ ํฌ์ŠคํŒ…์€ VGGNet ๊ด€๋ จ ํฌ์ŠคํŠธ์ž…๋‹ˆ๋‹ค. ๋จผ์ € ILSVRC (Imagenet Large Scale Visual Recognition Challenges)์ด...

[ํŒŒ์ดํ† ์น˜] ํŒŒ์ดํ† ์น˜๋กœ CNN ๋ชจ๋ธ์„ ๊ตฌํ˜„ํ•ด๋ณด์ž! (๊ธฐ์ดˆํŽธ + DataLoader ์‚ฌ์šฉ๋ฒ•)

MNIST ๋ฐ์ดํ„ฐ - CNN ์‹ค์Šต ์˜ค๋Š˜์€ MNIST ๋ฐ์ดํ„ฐ๋กœ Convolutional Neural Network(์ดํ•˜ CNN)์„ ๊ตฌํ˜„ํ•˜๊ณ  ๋Œ๋ ค๋ณด๋Š” ์‹œ๊ฐ„์„ ๊ฐ–๋„๋ก ํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค! ๋จผ์ €, CNN์€ ํฌ๊ฒŒ ์•„๋ž˜์™€ ๊ฐ™์€ ๊ตฌ์„ฑ์š”์†Œ๋กœ ์ด๋ฃจ์–ด์ ธ ์žˆ์Šต๋‹ˆ๋‹ค. ํ•ฉ์„ฑ๊ณฑ ์—ฐ์‚ฐ(Convolution) : ์ด๋ฏธ์ง€์˜ ํŠน์„ฑ์„ ์ถ”์ถœํ•˜๋Š” ๊ณ„์ธต ๋งฅ์Šคํ’€๋ง(Max Pooling)...

[Paper Review] An Image Is Worth 16x16 Words : Transformers for Image Recognition at Scale (Vision Transformer)

์„ ์ • ์ด์œ  ์•ˆ๋…•ํ•˜์„ธ์š”! ์˜ค๋Š˜ ๋…ผ๋ฌธ๋ฆฌ๋ทฐ, ์ฝ”๋“œ๋ฆฌ๋ทฐํ•ด๋ณผ ๋…ผ๋ฌธ์€ โ€œAn Image Is Worth 16x16 Words: Transformers for Image Recognition at Scaleโ€ ๋กœ, ์ปดํ“จํ„ฐ ๋น„์ „์—์„œ Transformer์™€ Attention์ด ์“ฐ์ด๊ฒŒ ๋œ ๊ฒฐ์ •์  ๊ณ„๊ธฐ(?)๊ฐ€ ๋œ ๋…ผ๋ฌธ์ž…๋‹ˆ๋‹ค. ์ตœ๊ทผ ์ด์ชฝ ๋ถ„์•ผ์— ๊ด€์‹ฌ์ด ๋งŽ๋‹ค ๋ณด๋‹ˆ ์˜ค๋Š˜์€ ...

[Paper Review] An Architecture Combining Convolutional Neural Network(CNN) and Support Vector Machine(SVM) for Image Classification

์˜ค๋Š˜ ๋ฆฌ๋ทฐ/๋ฒˆ์—ญ/๊ตฌํ˜„ํ•  ๋…ผ๋ฌธ์€ โ€œAbien Fred M. Agarapโ€ ์ €์ž๊ฐ€ ์“ด ๋…ผ๋ฌธ์œผ๋กœ, โ€œYichuan Tangโ€์˜ โ€œDeep Learning using Linear Support Vector Machinesโ€์„ ๋ณด๊ณ  inspired๋˜์–ด ์—ฐ๊ตฌํ•˜๊ฒŒ ๋˜์—ˆ๋‹ค๊ณ  ํ•œ๋‹ค. ํ•˜๋‹จ์˜ ์ฐธ๊ณ  ๋…ผ๋ฌธ ์†Œ์Šค์— ํ•ด๋‹น ๋…ผ๋ฌธ ๋งํฌ์™€ ์ด๋ฒˆ ๋…ผ๋ฌธ์˜ ๋งํฌ๋ฅผ ์ฒจ๋ถ€์˜€๋‹ค. (์ฐธ๊ณ ) ...

[ํŒŒ์ดํ† ์น˜] ํŒŒ์ดํ† ์น˜ ๊ธฐ์ดˆ ์š”์†Œ (Autograd๋ž€)

์ˆœ์ „ํŒŒ์™€ ์—ญ์ „ํŒŒ ์‹ ๊ฒฝ๋ง(Neural Network)์€ ์–ด๋–ค ์ž…๋ ฅ ๋ฐ์ดํ„ฐ์— ๋Œ€ํ•ด ์‹คํ–‰๋˜๋Š” ์ค‘์ฒฉ๋œ ํ•จ์ˆ˜๋“ค์˜ ์ง‘ํ•ฉ์ฒด์ž…๋‹ˆ๋‹ค. ์‹ ๊ฒฝ๋ง์„ ์•„๋ž˜ 2๋‹จ๊ณ„๋ฅผ ๊ฑฐ์ณ ํ•™์Šต๋ฉ๋‹ˆ๋‹ค : ์ˆœ์ „ํŒŒ(Forward Propagation) ์—ญ์ „ํŒŒ(Backward Propagation) Forward Propagation (์ˆœ์ „ํŒŒ) Forward Propagatio...

[ํŒŒ์ดํ† ์น˜] ํŒŒ์ดํ† ์น˜ ๊ธฐ์ดˆ ์š”์†Œ (ํ…์„œํŽธ)

์˜ค๋Š˜์€ ํŒŒ์ดํ† ์น˜๋ฅผ ๋‹ค๋ฃจ๊ธฐ ์œ„ํ•ด ์ค‘์š”ํ•œ ๊ธฐ์ดˆ ์ง€์‹๋“ค ์ค‘ ํ…์„œ์— ๋Œ€ํ•ด ๋‹ค๋ฃจ์–ด๋ณผ ์˜ˆ์ •์ž…๋‹ˆ๋‹ค. Source : https://hadrienj.github.io/posts/Deep-Learning-Book-Series-2.1-Scalars-Vectors-Matrices-and-Tensors/ ํ…์„œ ํ…์„œ๋Š” โ€˜๋ฐ์ดํ„ฐ๋ฅผ ํ‘œํ˜„ํ•˜๋Š” ๋‹จ์œ„โ€™์ž…๋‹ˆ๋‹ค. ...