Talk:MNIST database

This is the talk page for discussing improvements to the MNIST database article.
This is not a forum for general discussion of the article's subject.

Put new text under old text. Click here to start a new topic.
Please sign and date your posts by typing four tildes ( ~~~~ ).
New to Wikipedia? Welcome! Ask questions, get answers.

Article policies

Article milestones
Date	Process	Result
October 12, 2013	Peer review	Reviewed

A fact from this article appeared on Wikipedia's Main Page in the "Did you know?" column on September 19, 2013.

The text of the entry was: Did you know ... that the best error rate a computer program has gotten on the MNIST database of handwritten digits is 0.23 percent?

This article is of interest to the following WikiProjects:

WikiProject Articles for creation

(Rated C-class)

This article was reviewed by member(s) of WikiProject Articles for creation. The project works to allow users to contribute quality articles and media files to the encyclopedia and track their progress as they are developed. To participate, please visit the project page for more information.

C

This article has been rated as C-Class on the project's quality scale.


	This article was accepted on 27 August 2013.

WikiProject Computer Vision

This article is within the scope of WikiProject Computer Vision, a collaborative effort to improve the coverage of Computer Vision on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.

???

This article has not yet received a rating on the quality scale.

???

This article has not yet received a rating on the importance scale.

WikiProject Computing (Rated C-class)

This article is within the scope of WikiProject Computing, a collaborative effort to improve the coverage of computers, computing, and information technology on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.

C This article has been rated as C-Class on the project's quality scale.

??? This article has not yet received a rating on the project's importance scale.


	This article has been automatically rated by a bot or other tool because one or more other projects use this class. Please ensure the assessment is correct before removing the `\|auto=` parameter.
	An image has been requested for this article. Please remove the `needs-image` parameter once the image is added.

Questions[edit]

Half of the training set and half of the test set were taken from NIST's training dataset, while the other halves were taken from NIST's testing dataset..... is that 4 halves? Surely quarters or rephrase
There have been a number of scientific papers attempting to achieve the lowest error rate.... what no computers? just papers?
SVM? This isn't a common abbreviation .... explain please?

Hope this is helpful Victuallers (talk) 22:05, 31 August 2013 (UTC)

@Victuallers: The first sentence was rephrased. (Good catch...) SVM is wikilinked both places it appears in the article, but I changed it to be spelled out. Thanks! APerson (talk!) 11:52, 1 September 2013 (UTC)

This is not for DYK.... I'm just curious. What are digits? I guess this is 0-9 but does it include 12.1 or 12,1 as the French would write it. Can it tell 9,123 which is a real number less than ten in France from 9,123 which is an integer over 9 thousand in England? Is the French 7 supported... and programmers who write a zero with a slash? You may not know much about this but a picture showing the variation in digits that this database holds would be intriguing... Victuallers (talk) 13:45, 1 September 2013 (UTC)

It's just the digits 0 through 9. I'm trying to get a picture of the database from one of the gazillions of scientific papers about it; I'll probably add a picture soon. APerson (talk!) 22:46, 1 September 2013 (UTC)

Citation about error rate of 0.27%[edit]

Tomash, I just undid your removal of a citation since I felt that the achievement of a lowest error rate was important information. APerson (talk!) 18:13, 9 June 2014 (UTC)

@APerson:, there is another citation of a paper by the same authors two sentences ahead which states 0.23 error rate. So I removed the citation as it seemed to me that 0.27 was not the lowest error rate (0.27 > 0.23). Tomash (talk) 18:46, 9 June 2014 (UTC)

@Tomash: Reviewing the papers a lot more closely, it looks like the most recent paper (that is to say, the 2012 one) does not acknowledge the 2011 paper. While writing the article initially, I relied on the 2012 one for information and didn't look at the 2011 one until later, at which point I thought "Cool!" and stuck it in the lead. I'll move the 2011 paper down into the "Performance" section, since it did break the previous record in the literature at the time it was published. APerson (talk!) 01:09, 10 June 2014 (UTC)

0.21% Test error rate citation[edit]

I think the paper at http://yann.lecun.com/exdb/publis/pdf/wan-icml-13.pdf should be cited here, because they attained a test error rate of 0.21% in 2012 or 2013. The citation in the table just takes me to a set of 5 neural network weights on Google drive with a timestamp of November 2016. Searching for "Parallel Computing Center (Khmelnitskiy, Ukraine) represents an ensemble of 5 convolutional neural networks which performs on MNIST at 0.21 percent error rate." as per the citation gives me nothing.

Aizenberg et. al./0.0 error rate[edit]

From 2018-07-13 to 2018-07-23 and then (after reversion by Joel B. Lewis on 2018-07-27) again on 2018-08-07, one user added the Aizenberg et. al. result^[1] of 0.0% error rate to the page.

As Joel B. Lewis mentioned in his reversion, this is "[...] a month's worth of efforts to promote brand-new papers here with no evidence of significance or lasting impact". My removal is predicated on two arguments, one against the result and one against the edit.

- 0-example error rate is arguably impossible to reach on MNIST without overfitting on the test set (i.e. a serious methodological error in the paper); see for example the already-cited examples in the article^[2]. The only other paper that I'm aware of claiming something similar was withdrawn after finding exactly these issues. Therefore, arguably, the cited paper is wrong (but that's probably not for wikipedia to decide), but definitely Wikipedia:EXCEPTIONAL.

- The paper was edited into the article on the same day the conference on which it was published concluded. It has been added by an author with no contribution other than promoting this paper. It has been readded even after deletion. It therefore looks like WP:SPAM.

I have therefore deleted these additions. If I'm mistaken in this, please let me know.

Recurrent answer (talk) 10:18, 26 August 2018 (UTC)

"Decision Stream" Editing Campaign[edit]

This article has been targeted by an (apparent) campaign to insert "Decision Stream" into various Wikipedia pages about Machine Learning. "Decision Stream" refers to a recently published paper that currently has zero academic citations. ^[3] The number of articles that have been specifically edited to include "Decision Stream" within the last couple of months suggests conflict-of-interest editing by someone who wants to advertise this paper.

Known articles targeted:

ForgotMyPW (talk) 17:13, 2 September 2018 (UTC) (aka BustYourMyth)

^ I. Aizenberg and A. Gonzalez “Image Recognition using MLMVN and Frequency Domain Features”, Proceedings of the 2018 IEEE International Joint Conference on Neural Networks (IJCNN 2018), Rio De Janeiro, July, 2018, pp. 1550-1557.
^ MNIST classifier, GitHub. "Classify MNIST digits using Convolutional Neural Networks". Retrieved 3 August 2018.
^ Ignatov, D.Yu.; Ignatov, A.D. (2017). "Decision Stream: Cultivating Deep Decision Trees". IEEE ICTAI: 905–912. arXiv:1704.07657. doi:10.1109/ICTAI.2017.00140.

[:0-1] I. Aizenberg and A. Gonzalez “Image Recognition using MLMVN and Frequency Domain Features”, Proceedings of the 2018 IEEE International Joint Conference on Neural Networks (IJCNN 2018), Rio De Janeiro, July, 2018, pp. 1550-1557.

[mnist_github-2] MNIST classifier, GitHub. "Classify MNIST digits using Convolutional Neural Networks". Retrieved 3 August 2018.

[Decision_stream-3] Ignatov, D.Yu.; Ignatov, A.D. (2017). "Decision Stream: Cultivating Deep Decision Trees". IEEE ICTAI: 905–912. arXiv:1704.07657. doi:10.1109/ICTAI.2017.00140.

[1]

[2]

[3]

Talk:MNIST database

Contents

Questions[edit]

Citation about error rate of 0.27%[edit]

0.21% Test error rate citation[edit]

Aizenberg et. al./0.0 error rate[edit]

"Decision Stream" Editing Campaign[edit]

Navigation menu

Personal tools

Namespaces

Variants

Views

More

Search

Navigation

Interaction

Tools

Print/export

Languages