๋น„์ง€๋„ํ•™์Šต

๊ตฐ์ง‘ํ™”, ํ‘œํ˜„ ํ•™์Šต ๋“ฑ ๋น„์ง€๋„ ํ•™์Šต ๊ธฐ๋ฒ•.

๊ตฐ์ง‘ํ™”, ํ‘œํ˜„ ํ•™์Šต ๋“ฑ ๋น„์ง€๋„ ํ•™์Šต ๊ธฐ๋ฒ•.

์ƒ์œ„ ๊ฐœ๋…: ๋จธ์‹ ๋Ÿฌ๋‹

10 posts · ์•„์นด์ด๋ธŒ์—์„œ ํ•„ํ„ฐ๋กœ ๋ณด๊ธฐ

[NLP] 6. Topic Modeling์ด๋ž€?

1. Topic Modeling์ด๋ž€? ๋ณธ ๊ฐ•์˜๋Š” DSBA ๊ฐ•ํ•„์„ฑ ๊ต์ˆ˜๋‹˜์˜ ๊ฐ•์˜๋ฅผ ์ฐธ์กฐํ•˜์—ฌ ์ž‘์„ฑ๋˜์—ˆ์Šต๋‹ˆ๋‹ค. Topic Modeling์€ ๊ธฐ๊ณ„ ํ•™์Šต ๋ฐ ์ž์—ฐ์–ด ์ฒ˜๋ฆฌ ๋ถ„์•ผ์—์„œ ๋ฌธ์„œ ์ง‘ํ•ฉ ๋‚ด์—์„œ ์ž ์žฌ์ ์ธ ์ฃผ์ œ(Latent Topic)๋ฅผ ๋ฐœ๊ฒฌํ•˜๊ธฐ ์œ„ํ•ด ์‚ฌ์šฉํ•˜๋Š” ํ†ต๊ณ„์  ๋ชจ๋ธ๋ง ๊ธฐ๋ฒ•์ž…๋‹ˆ๋‹ค. ์ฃผ์–ด์ง„ ๋ฌธ์„œ์—์„œ ๋ฐ˜๋ณต์ ์œผ๋กœ ๋“ฑ์žฅํ•˜๋Š” ๋‹จ์–ด ํŒจํ„ด์„ ๋ถ„์„ํ•˜์—ฌ...

[CS294] Deep Unsupervised Learning: Introduction

๋‹ค์Œ์€ ์•„๋ž˜ โ€œBerkeley CS294 ๊ฐ•์˜โ€์— ๋Œ€ํ•œ ์š”์•ฝ ๋ฐ ํ•„๊ธฐ ๋‚ด์šฉ์„ ์ •๋ฆฌํ•œ ๊ฒƒ์ž…๋‹ˆ๋‹ค. ํ‹€๋ฆฐ ๋‚ด์šฉ์ด ์žˆ๋‹ค๋ฉด ๋Œ“๊ธ€ ๋ถ€ํƒ๋“œ๋ฆฝ๋‹ˆ๋‹ค ๐Ÿ™Œ Course : CS294-158 SP24 Deep Unsupervised Learning Instructor: Pieter Abbeel Lecture # : L1. Introduction ๋ณธ ๊ฐ•์˜๋Š”...

[์ดํƒ] ํ†ต๊ณ„ ๊ธฐ๋ฐ˜ ์ด์ƒํƒ์ง€ ์•Œ๊ณ ๋ฆฌ์ฆ˜

ํ†ต๊ณ„ ๊ธฐ๋ฐ˜ ์ด์ƒํƒ์ง€ ํ†ต๊ณ„์  ์ ‘๊ทผ ๋ฐฉ์‹์„ ์‚ฌ์šฉํ•˜์—ฌ ๋ชจ๋ธ์ด ๋ฐ์ดํ„ฐ์˜ โ€˜์ •์ƒ์ ์ธโ€™ ํ–‰๋™์„ ํ•™์Šตํ•˜๊ณ  ํ†ต๊ณ„์ ์œผ๋กœ ์ด๋ก€์ ์ธ ํ–‰๋™์„ ๋ณด์ด๋Š” ๋ฐ์ดํ„ฐ ํฌ์ธํŠธ๋ฅผ ์ด์ƒ์น˜๋กœ ์‹๋ณ„ํ•ฉ๋‹ˆ๋‹ค. GMM (Gaussian Mixture Models) ๋งํฌ: https://scikit-learn.org/stable/modules/generated/sklearn.mixture...

[์ดํƒ] ์•™์ƒ๋ธ” ๊ธฐ๋ฐ˜ ์ด์ƒํƒ์ง€ ์•Œ๊ณ ๋ฆฌ์ฆ˜

์•™์ƒ๋ธ” ๊ธฐ๋ฐ˜ ์ด์ƒํƒ์ง€ ์•™์ƒ๋ธ” ๊ธฐ๋ฐ˜ ์•Œ๊ณ ๋ฆฌ์ฆ˜์€ ์—ฌ๋Ÿฌ ๊ฐœ์˜ ์˜์‚ฌ๊ฒฐ์ • ๊ธฐ๋ฐ˜ ๋ชจ๋ธ์„ ์กฐํ•ฉํ•˜์—ฌ ์ด์ƒ์น˜๋ฅผ ํƒ์ง€ํ•ฉ๋‹ˆ๋‹ค. ์ด ๋ฐฉ๋ฒ•์€ ์ผ๋ฐ˜์ ์œผ๋กœ ๋†’์€ ์ •ํ™•๋„๋ฅผ ์ œ๊ณตํ•ฉ๋‹ˆ๋‹ค. Isolation Forest ๋งํฌ: https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.IsolationForest....

[์ดํƒ] ๋ฐ€๋„ ๊ธฐ๋ฐ˜ ์ด์ƒํƒ์ง€ ์•Œ๊ณ ๋ฆฌ์ฆ˜

๋ฐ€๋„ ๊ธฐ๋ฐ˜ ์ด์ƒํƒ์ง€ ๋ฐ€๋„ ๊ธฐ๋ฐ˜ ์•Œ๊ณ ๋ฆฌ์ฆ˜์€ ๋ฐ์ดํ„ฐ ํฌ์ธํŠธ ๊ฐ„์˜ ์ง€์—ญ์  ๋ฐ€๋„๋ฅผ ๊ธฐ๋ฐ˜์œผ๋กœ ์ด์ƒ์น˜๋ฅผ ํƒ์ง€ํ•ฉ๋‹ˆ๋‹ค. ์ด๋Ÿฌํ•œ ์•Œ๊ณ ๋ฆฌ์ฆ˜์€ ๋ฐ์ดํ„ฐ ๋ถ„ํฌ๊ฐ€ ๋ถˆ๊ท ์ผํ•œ ๊ฒฝ์šฐ์—๋„ ์ž˜ ์ž‘๋™ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. DBSCAN ๋งํฌ: https://scikit-learn.org/stable/modules/generated/sklearn.cluster.DBSCAN.html ...

[์ดํƒ] ๋‚ด๊ฐ€ ๋ณด๋ ค๊ณ  ์ •๋ฆฌํ•œ ML ์ด์ƒํƒ์ง€

์ด์ƒํƒ์ง€ ์ด์ƒํƒ์ง€๋Š” ๋ฐ์ดํ„ฐ์…‹์—์„œ ์ •์ƒ์ ์ธ ํŒจํ„ด์—์„œ ๋ฒ—์–ด๋‚œ ๋ฐ์ดํ„ฐ ํฌ์ธํŠธ๋ฅผ ์‹๋ณ„ํ•˜๋Š” ๊ณผ์ •์ž…๋‹ˆ๋‹ค. ์ด๋Ÿฌํ•œ ์ด์ƒ์น˜๋Š” ๋ฐ์ดํ„ฐ์˜ ๋ถ„ํฌ๋‚˜ ํŠน์„ฑ์—์„œ ๋ˆˆ์— ๋„๊ฒŒ ๋‹ค๋ฅธ ์ ๋“ค์„ ๋‚˜ํƒ€๋‚ด๋ฉฐ, ์ข…์ข… ์˜ค๋ฅ˜, ์‚ฌ๊ธฐ, ๊ฒฐํ•จ ๋“ฑ์˜ ๋ฌธ์ œ๋ฅผ ๋‚˜ํƒ€๋‚ผ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ๋”ฐ๋ผ์„œ ์ด์ƒํƒ์ง€๋Š” ๋ฐ์ดํ„ฐ ๋ถ„์„, ํ’ˆ์งˆ ๊ด€๋ฆฌ, ๋ณด์•ˆ ๋“ฑ ๋‹ค์–‘ํ•œ ๋ถ„์•ผ์—์„œ ์ค‘์š”ํ•œ ์—ญํ• ์„ ํ•ฉ๋‹ˆ๋‹ค. ์ด์ƒํƒ์ง€ ๋ฐฉ๋ฒ•๋ก ์— ๋”ฐ๋ฅธ ...

[์ดํƒ] ๊ฒฐ์ • ๊ฒฝ๊ณ„ ๊ธฐ๋ฐ˜ ์ด์ƒํƒ์ง€ ์•Œ๊ณ ๋ฆฌ์ฆ˜

๊ฒฐ์ • ๊ฒฝ๊ณ„ ๊ธฐ๋ฐ˜ ์ด์ƒํƒ์ง€ ๊ฒฐ์ • ๊ฒฝ๊ณ„ ๊ธฐ๋ฐ˜ ์•Œ๊ณ ๋ฆฌ์ฆ˜์€ ๋ฐ์ดํ„ฐ์˜ ๋ถ„ํฌ๋ฅผ ํ•™์Šตํ•˜์—ฌ ๋ช…ํ™•ํ•œ ๊ฒฝ๊ณ„๋ฅผ ํ˜•์„ฑํ•˜๊ณ , ์ด ๊ฒฝ๊ณ„๋ฅผ ๋ฒ—์–ด๋‚˜๋Š” ์ ๋“ค์„ ์ด์ƒ์น˜๋กœ ๊ฐ„์ฃผํ•ฉ๋‹ˆ๋‹ค. OCSVM (One-Class SVM) ๋งํฌ: https://scikit-learn.org/stable/modules/generated/sklearn.svm.OneClassSVM.html ...

[์ดํƒ] ๊ฑฐ๋ฆฌ ๊ธฐ๋ฐ˜ ์ด์ƒํƒ์ง€ ์•Œ๊ณ ๋ฆฌ์ฆ˜

๊ฑฐ๋ฆฌ ๊ธฐ๋ฐ˜ ์•Œ๊ณ ๋ฆฌ์ฆ˜ ๊ฑฐ๋ฆฌ ๊ธฐ๋ฐ˜ ์•Œ๊ณ ๋ฆฌ์ฆ˜์€ ๋ฐ์ดํ„ฐ ํฌ์ธํŠธ ๊ฐ„์˜ ๊ฑฐ๋ฆฌ๋ฅผ ๊ธฐ๋ฐ˜์œผ๋กœ ์ด์ƒ์น˜๋ฅผ ํƒ์ง€ํ•ฉ๋‹ˆ๋‹ค. ๋ฐ์ดํ„ฐ ํฌ์ธํŠธ๊ฐ€ ๋‹ค๋ฅธ ํฌ์ธํŠธ์™€์˜ ๊ฑฐ๋ฆฌ๊ฐ€ ๋ฉ€๋‹ค๋ฉด ์ด์ƒ์น˜๋กœ ๊ฐ„์ฃผ๋ฉ๋‹ˆ๋‹ค. K-Means Clustering ๊ธฐ๋ฐ˜ ์ด์ƒํƒ์ง€ ๋งํฌ: https://scikit-learn.org/stable/modules/generated/sklearn.cluster.K...

[๋จธ์‹ ๋Ÿฌ๋‹] ์ด์ƒ ํƒ์ง€ ๊ฐœ์š” ๋ฐ ๋ฐ€๋„ ๊ธฐ๋ฐ˜ ์ด์ƒ์น˜ํƒ์ง€

๋ณธ ํฌ์ŠคํŠธ๋Š” ๊ณ ๋ ค๋Œ€ํ•™๊ต ๊ฐ•ํ•„์„ฑ ๊ต์ˆ˜๋‹˜์˜ ๊ฐ•์˜๋ฅผ ์ˆ˜๊ฐ• ํ›„ ์ •๋ฆฌ๋ฅผ ํ•œ ๊ฒƒ์ž…๋‹ˆ๋‹ค. ์ž‘์„ฑ ๋ฐ ์„ค๋ช…์˜ ํŽธ์˜๋ฅผ ์œ„ํ•ด ์•„๋ž˜ ํฌ์ŠคํŠธ๋Š” ๋ฐ˜๋ง๋กœ ์ž‘์„ฑํ•œ ์  ์–‘ํ•ด๋ถ€ํƒ๋“œ๋ฆฝ๋‹ˆ๋‹ค. Abnormal Data๋ž€ Anomaly Data๋Š” ๋‹ค์Œ๊ณผ ๊ฐ™์ด Hawkins์™€ Harmeling์— ์˜ํ•ด ์ •์˜๋œ๋‹ค. Observations that deviate so much f...

[๋จธ์‹ ๋Ÿฌ๋‹] ๊ฑฐ๋ฆฌโ€ข๊ตฐ์ง‘โ€ข์„œํฌํŠธ๋ฒกํ„ฐ ๊ธฐ๋ฐ˜ ์ด์ƒํƒ์ง€ ๊ธฐ๋ฒ•

๋ณธ ํฌ์ŠคํŠธ๋Š” ๊ณ ๋ ค๋Œ€ํ•™๊ต ๊ฐ•ํ•„์„ฑ ๊ต์ˆ˜๋‹˜์˜ ๊ฐ•์˜๋ฅผ ์ˆ˜๊ฐ• ํ›„ ์ •๋ฆฌ๋ฅผ ํ•œ ๊ฒƒ์ž…๋‹ˆ๋‹ค. ์ž‘์„ฑ ๋ฐ ์„ค๋ช…์˜ ํŽธ์˜๋ฅผ ์œ„ํ•ด ์•„๋ž˜ ํฌ์ŠคํŠธ๋Š” ๋ฐ˜๋ง๋กœ ์ž‘์„ฑํ•œ ์  ์–‘ํ•ด๋ถ€ํƒ๋“œ๋ฆฝ๋‹ˆ๋‹ค. ๊ฑฐ๋ฆฌ ๊ธฐ๋ฐ˜ ์ด์ƒ์น˜ํƒ์ง€ K-Nearest Neighbor-based Anomaly Detection ๊ฐ ๋ฐ์ดํ„ฐ์— ๋Œ€ํ•œ Anomaly Score๋ฅผ K๊ฐœ์˜ ๊ทผ์ ‘ ์ด์›ƒ๊นŒ์ง€์˜ ๊ฑฐ๋ฆฌ๋ฅผ ์ด์šฉํ•˜์—ฌ ๊ณ„์‚ฐ...