1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
| 6.4 Adoption
The validity of design decisions and their impact on ease-of-use is hard to measure. As a proxy,
we tried to quantify how well the machine learning community received PyTorch by counting how
often various machine learning tools (including Caffe, Chainer, CNTK, Keras, MXNet, PyTorch,
TensorFlow, and Theano) are mentioned on arXiv e-Prints since the initial release of PyTorch in
January 2017. In Figure 3 we report the monthly number of mentions of the word "PyTorch" as a
percentage of all mentions among these deep learning frameworks. We counted tools mentioned
multiple times in a given paper only once, and made the search case insensitive to account for various
spellings.
Figure 3: Among arXiv papers each month that mention common deep learning frameworks, percentage of
them that mention PyTorch.
7 Conclusion and future work
PyTorch has become a popular tool in the deep learning research community by combining a focus
on usability with careful performance considerations. In addition to continuing to support the latest
trends and advances in deep learning, in the future we plan to continue to improve the speed and
scalability of PyTorch. Most notably, we are working on the PyTorch JIT: a suite of tools that
allow PyTorch programs to be executed outside of the Python interpreter where they can be further
optimized. We also intend to improve support for distributed computation by providing efficient
primitives for data parallelism as well as a Pythonic library for model parallelism based around
remote procedure calls.
8 Acknowledgements
We are grateful to the PyTorch community for their feedback and contributions that greatly influenced
the design and implementation of PyTorch. We thank all the PyTorch core team members, contributors
and package maintainers including Ailing Zhang, Alex Suhan, Alfredo Mendoza, Alican Bozkurt,
Andrew Tulloch, Ansha Yu, Anthony Shoumikhin, Bram Wasti, Brian Vaughan, Christian Puhrsch,
David Reiss, David Riazati, Davide Libenzi, Dmytro Dzhulgakov, Dwaraj Rajagopal, Edward Yang,
Elias Ellison, Fritz Obermeyer, George Zhang, Hao Lu, Hong Xu, Hung Duong, Igor Fedan, Ilia
Cherniavskii, Iurii Zdebskyi, Ivan Kobzarev, James Reed, Jeff Smith, Jerry Chen, Jerry Zhang, Jiakai
Liu, Johannes M. Dieterich, Karl Ostmo, Lin Qiao, Martin Yuan, Michael Suo, Mike Ruberry, Mikhail
Zolothukhin, Mingzhe Li, Neeraj Pradhan, Nick Korovaiko, Owen Anderson, Pavel Belevich, Peter
Johnson, Pritam Damania, Raghuraman Krishnamoorthi, Richard Zou, Roy Li, Rui Zhu, Sebastian
Messmer, Shen Li, Simon Wang, Supriya Rao, Tao Xu, Thomas Viehmann, Vincent Quenneville-
Belair, Vishwak Srinivasan, Vitaly Fedyunin, Wanchao Liang, Wei Yang, Will Feng, Xiaomeng Yang,
Xiaoqiang Zheng, Xintao Chen, Yangqing Jia, Yanli Zhao, Yinghai Lu and Zafar Takhirov.
References
[1]Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick,
Sergio Guadarrama, and Trevor Darrell. Caffe: Convolutional architecture for fast feature
embedding. arXiv preprint arXiv:1408.5093 , 2014.
[2]Frank Seide and Amit Agarwal. Cntk: Microsoft’s open-source deep-learning toolkit. In
Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery
and Data Mining , KDD ’16, pages 2135–2135, New York, NY , USA, 2016. ACM.
950%
40%
30%
20%
10%
0%
Jul2017
Jan2018
Jul2018
Jan2019
|
