Pending material

---

Long

• LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale (2022)
  • goal - quantized the model to speed up large model (with drawback)
• Neural Architectures for Named Entity Recognition(2016)
• Representation Learning for Information Extraction from Form-like Documents(2020)
• AN IMAGE IS WORTH 16X16 WORDS: TRANSFORMERS FOR IMAGE RECOGNITION AT SCALE (2021)
• DETR : https://arxiv.org/abs/2005.12872 (2020)
• Machine learning type - Online learning - https://arxiv.org/abs/1802.02871 (2018)
• Paper to explain gradient descent, optimizer https://arxiv.org/pdf/1609.04747.pdf (2017)
• Mercari MLOps - https://youtu.be/3fo5YyRqRII?t=570
• Minecraft VPT training - https://arxiv.org/abs/2206.11795 (2022)
• Check what topic of statistic need to be studied
  • from Chou : 初等統計學 抽樣調查 迴歸分析
• ViTDet - https://arxiv.org/abs/2203.16527 (2022)
  • https://github.com/facebookresearch/detectron2/tree/main/projects/ViTDet
• *Imagic: Text-Based Real Image Editing with Diffusion Models* (2022)
  • https://arxiv.org/abs/2210.09276
  • example - https://twitter.com/_akhaliq/status/1582175757153230849
• Donut
  • https://github.com/clovaai/donut

Short

• Federated Learning
  • https://pair.withgoogle.com/explorables/federated-learning/
• How large the data size should be - https://www.qualtrics.com/au/experience-management/research/determine-sample-size/?rid=ip&prevsite=en&newsite=au&geo=JP&geomatch=au
• GPT3 - https://openai.com/blog/gpt-3-apps/
• Image processing short tutorials
  • https://datacarpentry.org/image-processing/
• metric learning https://towardsdatascience.com/the-why-and-the-how-of-deep-metric-learning-e70e16e199c0
• sklearn - stratify - https://scikit-learn.org/stable/modules/cross_validation.html#stratification
• Model architecture
  • CNN, Rnn, GAN
• ViT (vision transformer)
• Line OCR source code
  • https://github.com/clovaai/donut
• Diff - normalize vs standardize vs regularization
  • Regularization
    • Regularization is a way to avoid model from overfitting. It has different kind of techniques, such as L2, dropout…etc https://www.reddit.com/r/learnmachinelearning/comments/w7yrog/what_regularization_does_to_a_machine_learning/?utm_medium=android_app&utm_source=share
• adversarial validation
• A guy got 2nd on kaggle competition - step by step guide - https://twitter.com/marktenenholtz/status/1539578965920083968
• MLFlows ← check how to use it
• wandb ← check this
• weak supervision learning ?
• inductive learning
  • refers to a learning algorithm that learns from labeled training data and generalizes to new data, such as a test dataset.
• transductive learning
  • refers to learning from labeled training data and generalizing to available unlabeled (training) data.
  • https://machinelearningmastery.com/transduction-in-machine-learning/
• MeanShift
  • the 2nd section here - https://www.christopherlovell.co.uk/blog/2016/07/04/image-pca-deckchair.html
• Semi supervised learning example on mincraft game - https://openai.com/blog/vpt/
  • Check its github src - https://github.com/openai/Video-Pre-Training
  • Video pretraining (vpt)
  • Inverse dynamics model (idm)
• Non maximum suppression ( CV)
• Gpu puzzle - an interactive notebook to learn gpu programming - https://github.com/srush/GPU-Puzzles

Comments