Paper Review

๋…ผ๋ฌธ์„ ์ฝ๊ณ  ํ•ต์‹ฌ ์•„์ด๋””์–ด์™€ ์‹คํ—˜์„ ์ •๋ฆฌํ•œ ๋ฆฌ๋ทฐ.

๋…ผ๋ฌธ์„ ์ฝ๊ณ  ํ•ต์‹ฌ ์•„์ด๋””์–ด์™€ ์‹คํ—˜์„ ์ •๋ฆฌํ•œ ๋ฆฌ๋ทฐ.

32 posts · ์•„์นด์ด๋ธŒ์—์„œ ํ•„ํ„ฐ๋กœ ๋ณด๊ธฐ

[Paper Review] K-EXAONE Technical Report

https://arxiv.org/abs/2601.01739 ๋„์ž…: ํ•œ๊ตญ AI ์ƒํƒœ๊ณ„์˜ ๋„์ „๊ณผ K-EXAONE์˜ ํƒ„์ƒ ๊ธ€๋กœ๋ฒŒ ๋Œ€๊ทœ๋ชจ ์–ธ์–ด ๋ชจ๋ธ(LLM) ๊ฐœ๋ฐœ ๊ฒฝ์Ÿ์ด ์น˜์—ดํ•ด์ง€๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค. Closed-Source ๋ชจ๋ธ์ด ์—ฌ์ „ํžˆ ์„ฑ๋Šฅ ์šฐ์œ„๋ฅผ ์ ํ•˜๊ณ  ์žˆ์ง€๋งŒ, Open-Weight ๋ชจ๋ธ๋“ค์ด ์ˆ˜์ฒœ์–ต ํŒŒ๋ผ๋ฏธํ„ฐ๋ฅผ ๋„˜์–ด ์กฐ(trillion) ๋‹จ์œ„ ์Šค์ผ€์ผ...

[Paper Review] LLaVA: Visual Instruction Tuning - ๋ฉ€ํ‹ฐ๋ชจ๋‹ฌ AI์˜ ์ƒˆ๋กœ์šด ํŒจ๋Ÿฌ๋‹ค์ž„

https://arxiv.org/pdf/2304.08485 ๋…ผ๋ฌธ ์ •๋ณด ์ œ๋ชฉ: Visual Instruction Tuning ์ €์ž: Haotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae Lee ์†Œ์†: University of Wisconsinโ€“Madison, Microsoft Research, ...

[Paper Review] LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models

https://arxiv.org/pdf/2403.15388 ๋…ผ๋ฌธ ์ •๋ณด ์ œ๋ชฉ: LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models ์ €์ž: Yuzhang Shang, Mu Cai, Bingxin Xu, Yong Jae Lee, Yan Yan ...

[Paper Review] MM-Groundung-DINO : An Open and Comprehensive Pipeline for Unified Object Grounding and Detection

https://arxiv.org/pdf/2401.02361 ๋ณธ ๋ฆฌ๋ทฐ๋Š” ์›๋ฌธ์„ ์ตœ๋Œ€ํ•œ ์ง์—ญํ•œ ๋‚ด์šฉ์ž…๋‹ˆ๋‹ค. ์—ฌ๊ธฐ์„œ โ€œ์šฐ๋ฆฌ๋Š”โ€์€ ์ €์ž๋ฅผ ์ง€์นญํ•ฉ๋‹ˆ๋‹ค. ์ฐธ๊ณ  ๋ถ€ํƒ๋“œ๋ฆฝ๋‹ˆ๋‹ค. ์ดˆ๋ก Grounding-DINO๋Š” Open-Vocabulary Detection (OVD), Phrase Grounding (PG), Referring Expression C...

[Paper Review] LLM-Det : Learning Strong Open-Vocabulary Object Detectors under the Supervision of Large Language Models

https://arxiv.org/pdf/2501.18954 ๋ณธ ๋ฆฌ๋ทฐ๋Š” ์›๋ฌธ์„ ์ตœ๋Œ€ํ•œ ์ง์—ญํ•œ ๋‚ด์šฉ์ž…๋‹ˆ๋‹ค. ์—ฌ๊ธฐ์„œ โ€œ์šฐ๋ฆฌ๋Š”โ€์€ ์ €์ž๋ฅผ ์ง€์นญํ•ฉ๋‹ˆ๋‹ค. ์ฐธ๊ณ  ๋ถ€ํƒ๋“œ๋ฆฝ๋‹ˆ๋‹ค. Abstract ์ตœ๊ทผ open-vocabulary detector๋“ค์€ ํ’๋ถ€ํ•œ region-level ์ฃผ์„ ๋ฐ์ดํ„ฐ๋กœ ์œ ๋งํ•œ ์„ฑ๋Šฅ์„ ๋‹ฌ์„ฑํ•˜๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค. (์ฐธ๊ณ ) Regio...

[Paper Review] RT-DETRv2: Improved Baseline with Bag-of-Freebies for Real-Time Detection Transformer

https://arxiv.org/pdf/2407.17140 ๋ณธ ๋ฆฌ๋ทฐ๋Š” ์›๋ฌธ์„ ์ตœ๋Œ€ํ•œ ์ง์—ญํ•œ ๋‚ด์šฉ์ž…๋‹ˆ๋‹ค. ์—ฌ๊ธฐ์„œ โ€œ์šฐ๋ฆฌ๋Š”โ€์€ ์ €์ž๋ฅผ ์ง€์นญํ•ฉ๋‹ˆ๋‹ค. ์ฐธ๊ณ  ๋ถ€ํƒ๋“œ๋ฆฝ๋‹ˆ๋‹ค. ์ดˆ๋ก ์ด ๋ณด๊ณ ์„œ์—์„œ๋Š” ๊ฐœ์„ ๋œ ์‹ค์‹œ๊ฐ„ Detection Transformer์ธ RT-DETRv2๋ฅผ ์ œ์‹œํ•ฉ๋‹ˆ๋‹ค. RT-DETRv2๋Š” ๊ธฐ์กด์˜ ์ตœ์‹  ์‹ค์‹œ๊ฐ„ detector์ธ RT-D...

[Paper Review] OmDet_Turbo : Real-time Transformer-based Open-Vocabulary Detection with Efficient Fusion Head

https://arxiv.org/pdf/2403.06892 ์ดˆ๋ก End-to-end transformer ๊ธฐ๋ฐ˜ detector (DETRs)๋Š” ์–ธ์–ด modality ํ†ตํ•ฉ์„ ํ†ตํ•ด closed-set๊ณผ open-vocabulary object detection (OVD) ์ž‘์—… ๋ชจ๋‘์—์„œ ๋›ฐ์–ด๋‚œ ์„ฑ๋Šฅ์„ ๋ณด์—ฌ์ฃผ์—ˆ์Šต๋‹ˆ๋‹ค. ๊ทธ๋Ÿฌ๋‚˜ ๋†’์€ ์—ฐ์‚ฐ ์š”๊ตฌ์‚ฌํ•ญ์œผ๋กœ...

[OpenAI] ์–ธ์–ด๋ชจ๋ธ ํ™˜๊ฐ(Hallucination) ํ˜„์ƒ: ์™œ AI๋Š” ํ™•์‹ ์— ์ฐฌ ๊ฑฐ์ง“๋ง์„ ํ• ๊นŒ?

Source: https://cdn.openai.com/pdf/d04913be-3f6f-4d2b-b283-ff432ef4aaa5/why-language-models-hallucinate.pdf https://openai.com/index/why-language-models-hallucinate/ ํ•ด๋‹น ๋ธ”๋กœ๊ทธ ํฌ์Šค...

[Paper Review] EXAONE Path 2.0: Pathology Foundation Model with End-to-End Supervision

https://arxiv.org/abs/2507.06639 1 PYEON, Myeongjang, et al. EXAONE Path 2.0: Pathology Foundation Model with End-to-End Supervision. arXiv preprint arXiv:2507.06639, 2025. Abstract ๋””์ง€ํ„ธ ๋ณ‘...

[Paper Review] EXAONE Deep: Reasoning Enhanced Language Models

https://arxiv.org/pdf/2503.12524 1 RESEARCH, L. G., et al. EXAONE Deep: Reasoning Enhanced Language Models. arXiv preprint arXiv:2503.12524, 2025. Abstract EXAONE Deep ์‹œ๋ฆฌ์ฆˆ๋ฅผ ์†Œ๊ฐœํ•ฉ๋‹ˆ๋‹ค. ์ด ๋ชจ๋ธ๋“ค์€ ...

[Paper Review] EXAONE 3.5: Series of Large Language Models for Real-world Use Cases

https://arxiv.org/pdf/2412.04862 1 AN, Soyoung, et al. EXAONE 3.5: Series of Large Language Models for Real-world Use Cases. arXiv e-prints, 2024, arXiv: 2412.04862. Abstract ์ด ๊ธฐ์ˆ  ๋ณด๊ณ ์„œ๋Š” LG A...

[Paper Review] Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond

https://arxiv.org/abs/2308.12966 1 WANG, Peng, et al. Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409.12191, 2024. ์ดˆ๋ก ๋ณธ ์—ฐ๊ตฌ...

[Paper Review] Qwen-Image Technical Report

https://arxiv.org/abs/2508.02324 1 WU, Chenfei, et al. Qwen-image technical report. arXiv preprint arXiv:2508.02324, 2025. ๋ณธ ์—ฐ๊ตฌ์—์„œ๋Š” ๋ณต์žกํ•œ ํ…์ŠคํŠธ ๋ Œ๋”๋ง๊ณผ ์ •๋ฐ€ํ•œ ์ด๋ฏธ์ง€ ํŽธ์ง‘์—์„œ ์ƒ๋‹นํ•œ ์ง„๋ณด๋ฅผ ์ด๋ฃฌ Qwen ์‹œ๋ฆฌ์ฆˆ์˜ ์ด๋ฏธ์ง€ ์ƒ์„ฑ fou...

[Paper Review] Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models

https://arxiv.org/abs/2311.07919 1 CHU, Yunfei, et al. Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models. arXiv preprint arXiv:2311.07919, 202...

[Paper Review] Qwen Technical Report

https://arxiv.org/abs/2309.16609 1 BAI, Jinze, et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023. ๐Ÿ’ก QWEN์€ ์ค‘๊ตญ์–ด๋กœ โ€œ์ฒœ ๊ฐœ์˜ ์งˆ๋ฌธโ€์„ ์˜๋ฏธํ•˜๋Š” Qianwen์˜ ๋ณ„๋ช…์ž…๋‹ˆ๋‹ค. โ€œQWENโ€์˜ ๋ฐœ์Œ์€ ๋งฅ๋ฝ๊ณผ ๋งํ•˜๋Š” ๊ฐœ์ธ์— ...

[Paper Review] EXAONE 4.0: Unified Large Language Models Integrating Non-reasoning and Reasoning Modes

https://arxiv.org/pdf/2507.11407 Introduction LLM ์ƒํƒœ๊ณ„์—์„œ ๊ฐ€์žฅ ๋šœ๋ ทํ•œ ํŠธ๋ Œ๋“œ ์ค‘ ํ•˜๋‚˜๋Š” โ€œ๋น ๋ฅธ ์‘๋‹ตโ€๊ณผ โ€œ๊นŠ์€ ์ถ”๋ก โ€์„ ํ•˜๋‚˜์˜ ๋ชจ๋ธ๋กœ ์ œ๊ณตํ•˜๋Š” Hybrid ๋ชจ๋ธ์˜ ๋ถ€์ƒ์ž…๋‹ˆ๋‹ค. ์—ฌ๊ธฐ์„œ โ€œ๋น ๋ฅธ ์‘๋‹ตโ€์ด๋ž€ ์ผ๋ฐ˜์ ์ธ ๋Œ€ํ™”๋‚˜ ์š”์•ฝ์ฒ˜๋Ÿผ ์ฆ‰๊ฐ์ ์ธ ๋‹ต๋ณ€์ด ํ•„์š”ํ•œ ์ƒํ™ฉ์„, โ€œ๊นŠ์€ ์ถ”๋ก โ€์ด๋ž€ ๋ณต์žกํ•œ ์ˆ˜ํ•™ ๋ฌธ์ œ๋‚˜...

[Paper Review] A generic non-invasive neuromotor interface for human-computer interaction

๋ฉ”ํƒ€(Meta)๊ฐ€ ์†๋ชฉ ๋ฐด๋“œ๋ฅผ ์ฐฉ์šฉํ•˜๊ณ  ์†๋ชฉ์ด๋‚˜ ์†๊ฐ€๋ฝ์˜ ์ž‘์€ ์›€์ง์ž„๋งŒ์œผ๋กœ๋„ ๋งˆ์šฐ์Šค ์กฐ์ž‘๊ณผ ํ‚ค๋ณด๋“œ ์ž…๋ ฅ์ด ๊ฐ€๋Šฅํ•œ ํ˜์‹ ์ ์ธ ๊ธฐ์ˆ ์„ ๋ฐœํ‘œํ–ˆ์Šต๋‹ˆ๋‹ค. ์ด ๊ธฐ์ˆ ์€ ์šฐ๋ฆฌ ๋ชธ์ด ๋งŒ๋“ค์–ด๋‚ด๋Š” ์•„์ฃผ ๋ฏธ์„ธํ•œ ์ „๊ธฐ์‹ ํ˜ธ๋ฅผ ์ฝ์–ด๋‚ด ์ปดํ“จํ„ฐ์™€ ์†Œํ†ตํ•˜๋Š” ๋ฐฉ์‹์„ ๊ทผ๋ณธ์ ์œผ๋กœ ๋ฐ”๊ฟ€ ์ž ์žฌ๋ ฅ์„ ๊ฐ€์ง€๊ณ  ์žˆ๋Š”๋ฐ์š”. ์†๊ฐ€๋ฝ์œผ๋กœ ํ—ˆ๊ณต์ด๋‚˜ ์ฑ…์ƒ ์œ„์— ๊ธ€์”จ๋ฅผ ์“ฐ๋ฉด ๊ทธ๋Œ€๋กœ ํ™”๋ฉด์— ์ž…๋ ฅ๋˜๋Š” ๋ฏธ๋ž˜๊ฐ€ ํ˜„์‹ค๋กœ...

[Paper Review] An Efficient Statistical Method for Image Noise Level Estimation

๋งํฌ : https://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Chen_An_Efficient_Statistical_ICCV_2015_paper.pdf (์ฐธ๊ณ ) ์ด ๋…ผ๋ฌธ์€ 2015๋…„์— ๋ฐœํ‘œ๋œ ๊ฒƒ์œผ๋กœ, ํ˜„์žฌ ๊ธฐ์ค€์—์„œ๋Š” ์ตœ์‹  ๊ธฐ์ˆ ์ด๋ผ๊ณ  ๋ณด๊ธฐ ์–ด๋ ต์Šต๋‹ˆ๋‹ค. ๋ณธ๋ฌธ์—์„œ ์–ธ๊ธ‰๋œ โ€œ์ตœ๊ทผโ€ ...

[TREND] ํŠธ๋ Œ์Šคํฌ๋จธ ์ดํ›„์˜ ์ฐจ์„ธ๋Œ€ ์•„ํ‚คํ…์ณ: MoE, SSM, RetNet, V-JEPA

2017๋…„, โ€œAttention is All You Needโ€๋ผ๋Š” ๋…ผ๋ฌธ๊ณผ ํ•จ๊ป˜ ๋“ฑ์žฅํ•œ ํŠธ๋žœ์Šคํฌ๋จธ(Transformer)๋Š” AI ๋ชจ๋ธ์˜ ํ˜์‹ ์ ์ธ ๋ณ€ํ™”๋ฅผ ์ด๋Œ์—ˆ์Šต๋‹ˆ๋‹ค. ํ˜„์žฌ, ๋Œ€ํ˜• ์–ธ์–ด ๋ชจ๋ธ(LLM)๊ณผ ์ƒ์„ฑ AI๋Š” ์–ธ์–ด, ๋น„๋””์˜ค, ์ด๋ฏธ์ง€ ๋“ฑ์˜ ๋ฐ์ดํ„ฐ ์ฒ˜๋ฆฌ์—์„œ ์••๋„์ ์ธ ์„ฑ๋Šฅ์„ ๋ฐœํœ˜ํ•˜๋ฉฐ ์šฐ๋ฆฌ ์‚ถ์˜ ๋‹ค์–‘ํ•œ ์˜์—ญ์—์„œ ํ™œ์šฉ๋˜๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค. ๊ทธ๋Ÿฌ๋‚˜ ํŠธ๋žœ์Šคํฌ๋จธ ๊ธฐ...

[Paper Review] NLP ๊ณต๋ถ€ํ•˜๋Š” ์‚ฌ๋žŒ์ด๋ผ๋ฉด ๊ผญ ์ฝ์–ด์•ผํ•˜๋Š” ๋…ผ๋ฌธ ๋Œ€์‹  ์ •๋ฆฌํ•ด๋“œ๋ฆฝ๋‹ˆ๋‹ค

โœ๏ธ ์ž์—ฐ์–ด ์ฒ˜๋ฆฌ(NLP)๋Š” ๋น ๋ฅด๊ฒŒ ๋ฐœ์ „ํ•˜๊ณ  ์žˆ๋Š” ๋ถ„์•ผ๋กœ, ์ˆ˜๋งŽ์€ ํš๊ธฐ์ ์ธ ์—ฐ๊ตฌ ๋…ผ๋ฌธ๋“ค์ด ๋งค๋…„ ๋ฐœํ‘œ๋˜๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค. ๋งŒ์•ฝ ์—ฌ๋Ÿฌ๋ถ„์ด NLP์— ์ฒ˜์Œ ๋ฐœ์„ ๋“ค์ด๊ฑฐ๋‚˜, ์—ฐ๊ตฌ๋ฅผ ๋” ๊นŠ์ด ์ดํ•ดํ•˜๊ณ ์ž ํ•œ๋‹ค๋ฉด, ๋‹ค์Œ์— ์†Œ๊ฐœํ•  ๋…ผ๋ฌธ๋“ค์ด ํ•ต์‹ฌ ๊ฐœ๋…๊ณผ ์ตœ๊ทผ ๋ฐœ์ „ ๋™ํ–ฅ์„ ํŒŒ์•…ํ•˜๋Š” ๋ฐ ํฐ ๋„์›€์ด ๋  ๊ฒƒ์ž…๋‹ˆ๋‹ค. ๋‹ค์Œ์€ ํ•ด๋‹น ๋ธ”๋กœ๊ทธ์—์„œ ์†Œ๊ฐœํ•˜๋Š” โ€œMust-Read Res...

[Paper Review] Mamba2 - Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality

๋งํฌ : https://arxiv.org/pdf/2405.21060 ๋‹ค์Œ์€ ๋…ผ๋ฌธ โ€œTransformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualityโ€์˜ ๊ฐ ์ฑ•ํ„ฐ๋ณ„๋กœ ์ž์„ธํ•œ ๋ฆฌ๋ทฐ ๋ฐ ์ •๋ฆฌ์ž…๋‹ˆ๋‹ค. 1. Introdu...

[Paper Review] A COMPREHENSIVE REVIEW OF YOLO ARCHITECTURES IN COMPUTER VISION: FROM YOLOV1 TO YOLOV8 AND YOLO-NAS

YOLO ๋ชจ๋ธ ์„œ๋ฒ ์ด ํŽ˜์ดํผ ๋งํฌ : https://www.mdpi.com/2504-4990/5/4/83 ๐Ÿ“– (์ฐธ๊ณ ) YOLO ๋ฒ„์ „์˜ ๋ฐœ์ „ ๐Ÿ’ก ๊ณต์‹ ๋ฒ„์ „: YOLO์˜ ์› ๊ฐœ๋ฐœ์ž์ธ Joseph Redmon์€ YOLOv3๊นŒ์ง€ ๊ฐœ๋ฐœํ–ˆ์Šต๋‹ˆ๋‹ค. ์ดํ›„ ๊ทธ๋Š” ์œค๋ฆฌ์  ์ด์œ ๋กœ ๊ฐ์ฒด ํƒ์ง€ ์—ฐ๊ตฌ๋ฅผ ์ค‘๋‹จํ–ˆ์Šต๋‹ˆ๋‹ค. Joseph Redmon์ด ๊ฐ์ฒด ํƒ...

[Paper Review] Mamba: Linear-Time Sequence Modeling with Selective State Spaces

์ตœ๊ทผ ๋”ฅ๋Ÿฌ๋‹ ์•„ํ‚คํ…์ฒ˜์˜ ์ค‘์‹ฌ์—๋Š” ํŠธ๋žœ์Šคํฌ๋จธ๊ฐ€ ์ž๋ฆฌ ์žก๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค. ๋Œ€๊ทœ๋ชจ ์–ธ์–ด ๋ชจ๋ธ(LLM)๋ฟ๋งŒ ์•„๋‹ˆ๋ผ, ๊ทธ๋ฆผ์„ ์ƒ์„ฑํ•˜๋Š” ๋ฐ ์“ฐ์ด๋Š” ๋””ํ“จ์ „ ๋ชจ๋ธ ๋˜ํ•œ ํŠธ๋žœ์Šคํฌ๋จธ ๊ตฌ์กฐ๋ฅผ ํ™œ์šฉํ•˜๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค. ์ด์™ธ์—๋„ ์‹œ๊ณ„์—ด ๋ถ„์„์ด๋‚˜ ์ถ”์ฒœ ์‹œ์Šคํ…œ๊ณผ ๊ฐ™์€ ๋‹ค์–‘ํ•œ ๋ถ„์•ผ์—์„œ ํŠธ๋žœ์Šคํฌ๋จธ๊ฐ€ ํ•ต์‹ฌ์ ์ธ ์—ญํ• ์„ ํ•˜๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค. ๊ทธ๋Ÿฌ๋‚˜ ํŠธ๋žœ์Šคํฌ๋จธ๋ฅผ ๋Œ€์ฒดํ•  ์ˆ˜ ์žˆ๋Š” ์ƒˆ๋กœ์šด ์•„ํ‚คํ…์ฒ˜๋ฅผ ๋ชจ์ƒ‰...

[Paper Review] Structured State Space Models for Deep Sequence Modeling

์‹œ๊ณ„์—ด ๋ฐ์ดํ„ฐ๋ฅผ ํšจ์œจ์ ์œผ๋กœ ์ฒ˜๋ฆฌํ•˜๋Š” ๋ฐฉ๋ฒ•์€ ์ง€๋‚œ ๋ช‡ ๋…„ ๋™์•ˆ ๋น ๋ฅด๊ฒŒ ๋ฐœ์ „ํ–ˆ์Šต๋‹ˆ๋‹ค. ํŠนํžˆ, CMU์— ๊ณ„์‹  Albert Gu ๊ต์ˆ˜๋‹˜์€ ๊ธด ์‹œ๊ณ„์—ด ์˜์กด์„ฑ(Long-Range Dependencies, LRDs)์„ ์ฒ˜๋ฆฌํ•˜๋Š” ๋ฐ ์ง‘์ค‘ํ•œ HiPPO(2020), LSSL(2021), ๊ทธ๋ฆฌ๊ณ  S4(2022)์™€ ๊ฐ™์€ ์—ฐ๊ตฌ๋“ค์„ ํ•˜๊ณ  ๊ณ„์‹ญ๋‹ˆ๋‹ค. ์ด๋ฒˆ ๊ธ€์—์„œ๋Š” ์—ฐ๊ตฌ์˜ ํ๋ฆ„...

[Paper Review] Resurrecting Recurrent Neural Networks for Long Sequences

๋…ผ๋ฌธ โ€œResurrecting Recurrent Neural Networks for Long Sequencesโ€๋Š” 25 Apr 2023์— publish๋˜์—ˆ์œผ๋ฉฐ, ICML 2023 OralPoster์— ๋ฐœํ‘œ๋œ ๋…ผ๋ฌธ์ž…๋‹ˆ๋‹ค. ํ•ด๋‹น ๋…ผ๋ฌธ์€ โ€œRecurrent Neural Networks (RNN)์˜ ์„ฑ๋Šฅ์„ ๋ณต์›ํ•˜์—ฌ ๊ธด ์‹œํ€€์Šค์—์„œ์˜ ํšจ์œจ์ ์ธ ํ•™์Šต๊ณผ ์ถ”๋ก โ€์„ ...

[Paper Review] ํ•ฉ์„ฑ ๋ฐ์ดํ„ฐ = ๋ชจ๋ธ ๋ถ•๊ดด?

๋“ค์–ด๊ฐ€๋ฉฐ ์ตœ๊ทผ ์ธ๊ณต์ง€๋Šฅ(AI) ๋ถ„์•ผ์—์„œ ๋งค์šฐ ํฅ๋ฏธ๋กœ์šด ๋‘ ๋…ผ๋ฌธ์ด Nature์— ๊ฒŒ์žฌ๋˜์–ด ํฐ ํ™”์ œ๋ฅผ ๋ชจ์œผ๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค. ์•„๋ž˜ ๋‘ ๋…ผ๋ฌธ์€ ๊ณตํ†ต์ ์œผ๋กœ AI ๋ชจ๋ธ์˜ ๋ฐœ์ „๊ณผ ๊ด€๋ จ๋œ ์ค‘๋Œ€ํ•œ ๋ฌธ์ œ๋ฅผ ๋‹ค๋ฃจ๊ณ  ์žˆ์œผ๋ฉฐ, AI ๊ธฐ์ˆ ์˜ ์žฅ๊ธฐ์ ์ธ ์œ„ํ—˜์„ฑ์„ ๊ฒฝ๊ณ ํ•˜๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค. AI models collapse when trained on recursively gener...

[Paper Review] An Image Is Worth 16x16 Words : Transformers for Image Recognition at Scale (Vision Transformer)

์„ ์ • ์ด์œ  ์•ˆ๋…•ํ•˜์„ธ์š”! ์˜ค๋Š˜ ๋…ผ๋ฌธ๋ฆฌ๋ทฐ, ์ฝ”๋“œ๋ฆฌ๋ทฐํ•ด๋ณผ ๋…ผ๋ฌธ์€ โ€œAn Image Is Worth 16x16 Words: Transformers for Image Recognition at Scaleโ€ ๋กœ, ์ปดํ“จํ„ฐ ๋น„์ „์—์„œ Transformer์™€ Attention์ด ์“ฐ์ด๊ฒŒ ๋œ ๊ฒฐ์ •์  ๊ณ„๊ธฐ(?)๊ฐ€ ๋œ ๋…ผ๋ฌธ์ž…๋‹ˆ๋‹ค. ์ตœ๊ทผ ์ด์ชฝ ๋ถ„์•ผ์— ๊ด€์‹ฌ์ด ๋งŽ๋‹ค ๋ณด๋‹ˆ ์˜ค๋Š˜์€ ...

[Paper Review] An Architecture Combining Convolutional Neural Network(CNN) and Support Vector Machine(SVM) for Image Classification

์˜ค๋Š˜ ๋ฆฌ๋ทฐ/๋ฒˆ์—ญ/๊ตฌํ˜„ํ•  ๋…ผ๋ฌธ์€ โ€œAbien Fred M. Agarapโ€ ์ €์ž๊ฐ€ ์“ด ๋…ผ๋ฌธ์œผ๋กœ, โ€œYichuan Tangโ€์˜ โ€œDeep Learning using Linear Support Vector Machinesโ€์„ ๋ณด๊ณ  inspired๋˜์–ด ์—ฐ๊ตฌํ•˜๊ฒŒ ๋˜์—ˆ๋‹ค๊ณ  ํ•œ๋‹ค. ํ•˜๋‹จ์˜ ์ฐธ๊ณ  ๋…ผ๋ฌธ ์†Œ์Šค์— ํ•ด๋‹น ๋…ผ๋ฌธ ๋งํฌ์™€ ์ด๋ฒˆ ๋…ผ๋ฌธ์˜ ๋งํฌ๋ฅผ ์ฒจ๋ถ€์˜€๋‹ค. (์ฐธ๊ณ ) ...

[Paper Review] Transferring Inductive Bias Through Knowledge Distillation - (3/3)

์•ˆ๋…•ํ•˜์„ธ์š” :) ์˜ค๋Š˜์€ ์ง€๋‚œ๋ฒˆ ํฌ์ŠคํŒ…์— ์ด์–ด์„œ โ€œTransferring Inductive Bias Through Knowledge Distillationโ€ ๋…ผ๋ฌธ์— ๋Œ€ํ•œ ์ •๋ฆฌ๋ฅผ ์ด์–ด๋‚˜๊ฐ€ ๋ณด๋„๋ก ํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค. ์ด์ „ ํฌ์ŠคํŒ…์—์„œ ๋ณธ ๋…ผ๋ฌธ์—์„œ ๋‹ค๋ฃจ๊ฒŒ ๋  ์ฃผ์š” ๊ฐœ๋…๋“ค์ธ Knowledge Distillation๊ณผ Inductive Bias์— ๋Œ€ํ•œ ์„ค๋ช…๊ณผ RNNs v...

[Paper Review] Visualizing Data using t-SNE

์„ ์ • ์ด์œ  ์˜ค๋Š˜ ๋ฆฌ๋ทฐ/๋ฒˆ์—ญ/๊ตฌํ˜„ํ•  ๋…ผ๋ฌธ์€ Visualizing Data using t-SNE์œผ๋กœ, 2008๋…„์— Geoffrey Hinton์ด ์ €์ž์ธ ๋…ผ๋ฌธ์ž…๋‹ˆ๋‹ค. t-SNE(t-Stochastic Nearest Neighbor)์€ ์ง€๊ธˆ๊นŒ์ง€๋„ ์‹œ๊ฐํ™”๋ฅผ ํ•˜๋Š” ๋ฐ ์ž์ฃผ ์‚ฌ์šฉ๋˜๋Š” ์•Œ๊ณ ๋ฆฌ์ฆ˜์ž…๋‹ˆ๋‹ค. ์ด๋Š” ๊ณ ์ฐจ์›์˜ ๋ฒกํ„ฐ๋กœ ํ‘œํ˜„๋˜๋Š” ๋ฐ์ดํ„ฐ ๊ฐ„์˜ neighbor st...

[Paper Review] Transferring Inductive Bias Through Knowledge Distillation - (2/3)

์•ˆ๋…•ํ•˜์„ธ์š” :) ์˜ค๋Š˜์€ ์ง€๋‚œ๋ฒˆ ํฌ์ŠคํŒ…์— ์ด์–ด์„œ โ€œTransferring Inductive Bias Through Knowledge Distillationโ€ ๋…ผ๋ฌธ์— ๋Œ€ํ•œ ์ •๋ฆฌ๋ฅผ ์ด์–ด๋‚˜๊ฐ€ ๋ณด๋„๋ก ํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค. ์ด์ „ ํฌ์ŠคํŒ…์—์„œ ๋ณธ ๋…ผ๋ฌธ์—์„œ ๋‹ค๋ฃจ๊ฒŒ ๋  ์ฃผ์š” ๊ฐœ๋…๋“ค์ธ Knowledge Distillation๊ณผ Inductive Bias์— ๋Œ€ํ•œ ์„ค๋ช…์„ ํ•ด๋ณด์•˜๋Š”๋ฐ์š”...

[Paper Review] Transferring Inductive Bias Through Knowledge Distillation - (1/3)

์•ˆ๋…•ํ•˜์„ธ์š” :) ์˜ค๋Š˜ ๋ธ”๋กœ๊ทธ ํฌ์ŠคํŒ…์œผ๋กœ ๋‹ค๋ค„๋ณผ ๋‚ด์šฉ์€ ์–ผ๋งˆ ์ „์— ํฅ๋ฏธ๋กญ๊ฒŒ ์ฝ์–ด๋ณด์•˜๋˜ โ€œTransferring Inductive Bias Through Knowledge Distillationโ€์ด๋ผ๋Š” ๋…ผ๋ฌธ์ธ๋ฐ์š”! ํ•ด๋‹น ๋…ผ๋ฌธ์€ Knowledge Distillation์„ ์ด์šฉํ•˜์—ฌ ๊ณผ์—ฐ Inductive Bias๋ฅผ ์ „๋‹ฌํ•  ์ˆ˜ ์žˆ์„ ๊นŒ๋ฅผ ๋‹ค๋ฃฌ ๋…ผ๋ฌธ์ž…๋‹ˆ๋‹ค. ์•„์‰ฝ...