Measure Zero

Bayes 优化简介

2022-01-01 | ~ 2022-02-05 | Machine Learning

首先强烈推荐

Frazier, P. I. (2018). A tutorial on Bayesian optimization. arXiv Preprint arXiv:1807.02811.

写得非常清楚. 本文只简单介绍最基本的内容.

考虑优化问题

\[\max_{x\in A} f(x),\]

Bayesian optimization is designed for black-box derivative-free global optimization. 黑箱意思是不知道函数 $f$ 的形式和性质 (凸性等), 只能通过输入 $x$ 得到输出 $f(x)$, 另外也不知道导数信息, 目标是求解全局最优.

半监督学习简要

2021-12-31 | ~ | Machine Learning

参考

Zhu, X., & Goldberg, A. B. (2009). Introduction to semi-supervised learning. Synthesis lectures on artificial intelligence and machine learning, 3(1), 1-130.

书比较老, 介绍了 SVM 时代一些分类问题机器学习算法的半监督版本. 特色是强调半监督学习有效需要的假设, 以及不符合假设的人造数据样例的可视化展示. 本文只涉及通用的算法.

BERT 复习

2021-12-15 | ~ | Machine Learning

复习

Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv Preprint arXiv:1810.04805.

文档目录抽取

2021-12-14 | ~ 2022-10-19 | Machine Learning

文档结构化是很暧昧的词, 它可能的意思很多, 不过本文只考虑目录抽取.

结构化文档由各级章节标题和段落等逻辑结构组成, 比如对 HTML 来说, 逻辑结构包括 <body> <h1> <p> 等标签. 文档结构化任务基本等价于目录抽取, 因为识别出标题后剩下的就是段落. 这个领域可供搜索的关键词包括 document structure recognition, document layout analysis (版面分析) 等. 意义: 便于抽取信息, 高度定制化的展示等.

Transformer 复习

2021-12-10 | ~ | Machine Learning

复习

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 5998–6008.

[半监督] 非监督数据增强 (UDA)

2021-12-06 | ~ | Machine Learning

来自 Google 的

Xie, Q., Dai, Z., Hovy, E., Luong, M.-T., & Le, Q. V. (2019). Unsupervised data augmentation for consistency training. arXiv Preprint arXiv:1904.12848. [Code] [Code for PyTorch]

Consistency training methods simply regularize model predictions to be invariant to small noise applied to either input examples or hidden states. This framework makes sense intuitively because a good model should be robust to any small change in an input example or hidden states.