Repository: YBIGTA/DeepNLP-Study Branch: master Commit: 5a0f218593c7 Files: 8 Total size: 6.7 KB Directory structure: gitextract_i_057m3r/ ├── .gitignore ├── CNN_sentence/ │ └── README.md ├── LICENSE ├── QRNN/ │ └── README.md ├── README.md ├── TCML/ │ └── README.md └── show-attend-and-tell-pytorch/ ├── README.md └── models.py ================================================ FILE CONTENTS ================================================ ================================================ FILE: .gitignore ================================================ # Byte-compiled / optimized / DLL files __pycache__/ *.py[cod] *$py.class # C extensions *.so # Distribution / packaging .Python env/ build/ develop-eggs/ dist/ downloads/ eggs/ .eggs/ lib/ lib64/ parts/ sdist/ var/ wheels/ *.egg-info/ .installed.cfg *.egg # PyInstaller # Usually these files are written by a python script from a template # before PyInstaller builds the exe, so as to inject date/other infos into it. *.manifest *.spec # Installer logs pip-log.txt pip-delete-this-directory.txt # Unit test / coverage reports htmlcov/ .tox/ .coverage .coverage.* .cache nosetests.xml coverage.xml *.cover .hypothesis/ # Translations *.mo *.pot # Django stuff: *.log local_settings.py # Flask stuff: instance/ .webassets-cache # Scrapy stuff: .scrapy # Sphinx documentation docs/_build/ # PyBuilder target/ # Jupyter Notebook .ipynb_checkpoints # pyenv .python-version # celery beat schedule file celerybeat-schedule # SageMath parsed files *.sage.py # dotenv .env # virtualenv .venv venv/ ENV/ # Spyder project settings .spyderproject .spyproject # Rope project settings .ropeproject # mkdocs documentation /site # mypy .mypy_cache/ # macOS .DS_store .DS_store? ================================================ FILE: CNN_sentence/README.md ================================================ # A Sensitivity Analysis of (and Practitioners' Guide to) Convolutional Neural Networks for Sentence Classification ## 1. Background ### CNN Architecture begin with a tokenized sentence -> a sentence matrix sentence matrix의 row는 word vector들. word vector들은 pretrained 사용 word vectors dimension: d length of a given sentence: s *sentence matrix: s x d* treat sentence matrix as an ‘image’ perform convolution on it via linear filters filters with widths equal to the dimensionality of the word vectors (d) height of the filter: the region size of the filter. denote the sentence matrix by A A[i:j]: the sub-matrix of A from row i to row j. w: weight matrix b: bias term The output sequence of the convolution operator: $o_i = w \cdot{A[i:j+h-1]}$ feature map: $c_i = f(o_i + b)$ $c \in R^{s-h+1}$ A pooling function is thus applied to each feature map to induce a fixed-length vector. A common strategy is 1-max pooling the outputs generated from each filter map can be concatenated into a fixed-length, ‘top-level’ feature vector then fed through a softmax function to generate the final classification. ## 2. Baseline Models ![Imgur](https://i.imgur.com/s7aXR1N.png) **CNN architecture for sentence classification** region sizes -> gram feature map -> convolution 결과 필터 개수만큼의 feature map을 만들고, Max-pooling 과정을 거쳐 클래스 개수만큼의 스코어를 출력 We kept only the most frequent 30k n-grams for all datasets tuned hyperparameters via nested cross-fold validation incorporating word2vec embeddings into feature vectors ### configuration a linear kernel SVM exploiting uni- and bi-gram features used averaged word vectors (from Google word2vec or GloVe) ![Imgur](https://i.imgur.com/L7FZiC9.png) ## references A Sensitivity Analysis of (and Practitioners’ Guide to) Convolutional Neural Networks for Sentence Classification(2016) - Ye Zhang, Byron C. Wallace http://www.wildml.com/2015/11/understanding-convolutional-neural-networks-for-nlp/ https://ratsgo.github.io/natural%20language%20processing/2017/03/19/CNN/ ================================================ FILE: LICENSE ================================================ MIT License Copyright (c) 2017 YBIGTA Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ================================================ FILE: QRNN/README.md ================================================ https://arxiv.org/abs/1611.01576 ================================================ FILE: README.md ================================================ # DeepNLP-Study 딥러닝 기반 자연어처리 스터디의 논문 구현 코드 및 스터디 자료 모음 공간입니다. ## 2018년 7~8월 스터디 - 목표 - [CMU Neural Networks for NLP 2018](http://phontron.com/class/nn4nlp2017/index.html) 강의를 듣고 발제한다. - 매주 모든 사람이 두 강의씩 들으며, 그 중 두 명이 한 강의씩 맡아 발제한다. - 발제하지 않는 나머지 사람들은 질문을 하고 발제자는 답변을 한다. - 강의마다 나온 질문과 답변은 다음 위키 페이지에 모아서 정리한다. - [Q&A-for-CMU-Neural-Networks-for-NLP-2018](https://github.com/YBIGTA/DeepNLP-Study/wiki/%5BQ&A-for-CMU-Neural-Networks-for-NLP-2018%5D) - 참여자 : 김우정, 김지혁, 박성남, 조용래 # 과거 스터디 히스토리 ## 2018년 1~2월 스터디 목표 - 목표 - 이론 - 조재민님의 챗봇을 위한 딥러닝 강의 자료 공부하기 (논문 발제 모음 : [PAPERS REVIEWED](https://github.com/YBIGTA/DeepNLP-Study/wiki/%5BPAPERS-REVIEWED%5D)) - [Day 01 Introduction to Chatbot](https://www.slideshare.net/JaeminCho6/dl-chatbot-seminar-day-01-80593691) - [Day 02 Text Classification with CNN/RNN](https://www.slideshare.net/JaeminCho6/dl-chatbot-seminar-day-02) - [Day 03 Conversation Modeling with Seq2Seq / Attention](https://www.slideshare.net/JaeminCho6/dl-chatbot-seminar-day-03) - [Day 04 QA with External Memory](https://www.slideshare.net/JaeminCho6/dl-chatbot-seminar-day-04) - 구현 - 직접 딥러닝 모델을 한 달에 하나씩 구현해 보기 - [논문 구현 프로젝트](https://github.com/YBIGTA/DeepNLP-Study/projects/1?) - [스터디 세부 계획](https://github.com/YBIGTA/DeepNLP-Study/wiki/%5B%EC%8A%A4%ED%84%B0%EB%94%94-%EC%84%B8%EB%B6%80-%EA%B3%84%ED%9A%8D-(2018%EB%85%84-1~2%EC%9B%94)%5D) ## Github 운영 방식 * 모든 스터디 자료는 이 repository를 통해 공유하는 것을 원칙으로 합니다. * Wiki * 스터디 계획, 일정 등의 문서를 작성합니다. * 발제를 위한 논문 요약 및 설명 등도 공유합니다. * Projects * 논문 구현 프로젝트 진행 상황을 공유합니다. * 스터디 관련 전반적인 할 일들을 관리합니다. * Code * 논문 구현 코드를 논문 별로 폴더를 만들어서 commit합니다. * Commit message 대괄호 안에 논문 이름 약어를 적어 commit을 구분합니다. * 수정 사항을 영어로 적을 때는 명령문으로, 첫 글자는 대문자로 적습니다. * 두 명 이상의 사람이 동시에 같은 파일을 수정하지 않도록 주의합니다. * Commit message 예시 : `[논문이름] Do something` ================================================ FILE: TCML/README.md ================================================ # TCML Meta-Learning with Temporal Convolutions [https://arxiv.org/abs/1707.03141](https://arxiv.org/abs/1707.03141) ================================================ FILE: show-attend-and-tell-pytorch/README.md ================================================ # Show Attend and Tell paper: https://arxiv.org/abs/1502.03044 reference: https://github.com/yunjey/show-attend-and-tell ================================================ FILE: show-attend-and-tell-pytorch/models.py ================================================ from torchvision.models import vgg from torch import nn ## Encoder class Encoder(nn.Module): def __init__(self): super(Encoder, self).__init__() self.original_model = vgg.vgg19_bn(pretrained=True) self.convs = list(self.original_model.children())[0] self.layers = nn.Sequential(*list(self.convs)[:-1]) def forward(self, x): x = self.layers(x) return x