Talk:Pattern recognition

Overview section

I really like the first three paragraphs of the overview section. I'd like to remove the rest though, because it is only partially related. Less is more. Or we need more high-quality paragraphs here. I'll let this comment sink in for a while, I've done enough edits today (and tried to justify each one), let's see how many will be undone. Hobbes (talk) 20:38, 7 May 2009 (UTC)

Recognition vs. Analysis

Is there a difference between pattern analysis and pattern recognition?--Adoniscik (talk) 21:20, 28 December 2007 (UTC)

Unsupervised learning

Should this include unsupervised methods, too? E.g., clustering has major applications for pattern recognition. -- 16:55, 25 Mar 2005 (MET)

I disagree --- that material is already covered in unsupervised learning. Both supervised and unsupervised learning are covered in a high-level way at machine learning. -- hike395 17:37, 25 Mar 2005 (UTC)

Link to peer reviewed paper

Hi, I recently added some new information regarding the comparison of various classification techniques with a reference to a peer-reviewed article (van der Walt and Barnard). There seems to be some controversy on this subject, the link has been removed several times. I am currently doing my PhD on this topic and I know the information is very relevant.

Is the reference and the external link http://www.patternrecognition.co.za suitable for this site? If not, what can I do so that this information is not repeatedly removed? -cvdwalt

As a scholar, you should know that you may give references to strengthen your claims that are detailed in your article, and not add unexpected references without any prior presentation. Your self-promotion is then, in my opinion, irrelevant both in this article and in Wikipedia. --131.254.15.97 (talk) 14:33, 24 March 2009 (UTC)

I agree with the previous comment. Furthermore, the research mentioned is very, very specific and should maybe be moved to another page that could be linked from here. I also recall several other researchers that have dealt with this topic (e.g. T.K. Ho in her ICPR2004 paper, but there are others), which IMHO also speaks for a separate page. Hobbes (talk) 20:25, 7 May 2009 (UTC)

Merging/reorganizing Pattern recognition and statistical classification

Merge with statistical classification somehow? -- hike395 08:25, 10 Mar 2005 (UTC)

Done. Made this the main article. -- hike395 15:42, 10 Mar 2005 (UTC)

Why has statistical classification been redirected to pattern recognition of all places? Statistical classification far pre-dates pattern recognition or machine learning and has uses far beyond either. I think it is more properly a category within statistics and/or cladistics/taxonomy than pattern recognition. A link to it is warranted from this page, but why the redirect? -- User:Jrbouldin

Take a look at [1], which was the old statistical classification article before the merge. It was entirely about statistical classification algorithms, which is the subject of this article and has more detail here. If you think there is more material in statistical classification beyond the material here (or in supervised learning), feel free to write something up. We can also refactor it into the right spot. If you want to write a draft (and not make an "official" article), you can write something at statistical classification/temp, and we can discuss what to do with it. It's up to you. --- hike395 June 29, 2005 22:32 (UTC)

Thank you for the clarification and other advice. I am new but will give it a shot. Seems to me the topic needs to made more general.Jeeb 1 July 2005 01:39 (UTC)

Please see Talk:statistical classification for a continuation of this discussion. Thanks! hike395

Human pattern recognition?

Numerology (for example) links here and yet this article is entirely about pattern recognition as performed by machines. Surely humans recognize patterns too. Recury 21:00, 3 July 2006 (UTC)

Yes, how about some information on patern recognition in humans? 66.24.195.123 12:39, 19 August 2006 (UTC)

To point you in the direction, an example is how we hear words in music played backwards —Preceding unsigned comment added by 24.5.91.222 (talk) 01:33, 14 December 2009 (UTC)

Hidden Messages sidebar

"Hidden messages" sidebar is out of main topic of "Pattern recognition". I see no need to have in such a dominant place links to themes like Reverse speech or Numerology. I suggest removing sidebar "Hidden messages". Msm 10:51, 13 March 2007 (UTC)

No one was against my suggestion in a week time, so I remove hidden messages because of reasons written above. Msm 22:47, 20 March 2007 (UTC)

I also object to it so I posted a message on its talk page since it's back again. Let's form a consensus so we don't add/delete it repeatedly.--Adoniscik (talk) 06:35, 2 January 2008 (UTC)

Rule of three

Someone links to Rule of three (writing) in the See Also section. I see no relevance so I'll remove it. Could someone give justification here before putting it back in. --Offput 15:40, 23 March 2007 (UTC)

Contradiction

Huge contradiction in the introduction of the article: techniques of a sub-domain of automatic learning cannot rely on a priori knowledge! Do you know what you speak of or just quote on scholars? —Preceding unsigned comment added by 137.108.145.250 (talk) 10:15, 19 September 2007 (UTC)

major rewrite

I just rewrote the article almost totally (except for a couple of sections at the end), to reflect what I believe "pattern recognition" to actually be. The former article basically just described the process of classification, with a bit of other stuff thrown in. As used in books such as Christopher Bishop "Pattern Recognition and Machine Learning" (more or less the "bible" nowadays), pattern recognition clearly refers to any sort of automatic labeling of input data: Not just classifying data into one of a fixed set of classes, but clustering the data (including learning the number of clusters from the data), regression (producing a real-valued label), sequence labeling such as using an HMM or Kalman filter, etc. Bishop does not explicitly discuss syntactic parsing, but this would clearly also qualify as a type of pattern recognition. Benwing (talk) 06:40, 5 October 2010 (UTC)

Category:Pattern recognition

I think it should be created the category "Category:Pattern recognition" to put there all types of pattern recognition, like in the Spanish edition of Wikipedia ("es:Categoría:Reconocimiento de patrones"). What do you think?--Aliuken (talk) 19:51, 31 May 2011 (UTC)

Problem statement for unsupervised learning

The student in me was pleased to find the formal problem statement for the supervised case (although I must now learn it!). Can anyone add or cite a formal problem statement for the unsupervised case? I also added the link to Deep Learning which seems to fit well under unsupervised categorisation - trust I've not stepped on anyone's toes. P.r.newman (talk) 20:10, 5 March 2012 (UTC)

Disputed

In machine learning, pattern recognition is the assignment of a label to a given input value.

This is the definition of classification, and maybe of regression for a suitably broad definition of "label", not pattern recognition. Pattern recognition is much broader, to the point that it is practically synonymous with machine learning:

Pattern recognition has its origins in engineering, whereas machine learning grew out of computer science. However, these activities can be viewed as two facets of the same field, and together they have undergone substantial development over the past ten years. (p. vii)

The field of pattern recognition is concerned with the automatic discovery of regularities in data through the use of computer algorithms and with the use of these regularities to take actions such as classifying the data into different categories. (p. 1; this could be a better definition)

In other pattern recognition problems, the training data consists of a set of input vectors x without any corresponding target values. The goal in such unsupervised learning problems may be to discover groups of similar examples within the data, where it is called clustering, or to determine the distribution of data within the input space, known as density estimation, or to project the data from a high-dimensional space down to two or three dimensions for the purpose of visualization. Finally, the technique of reinforcement learning [...] is concerned with the problem of finding suitable actions to take in a given situation in order to maximize a reward. Here the learning algorithm is not given examples of optimal outputs, in contrast to supervised learning, but must instead discover them by a process of trial and error. (p. 3)

Quotes from Bishop, Christopher M. (2006). Pattern Recognition and Machine Learning. Springer. The rest of that books uses pattern recognition and machine learning interchangeably, making no distinction between the two apart from the one I quoted (unless I missed it; it's a big book, but I read quite a lot of it). We might as well merge this article with machine learning. QVVERTYVS (hm?) 16:55, 5 May 2014 (UTC)

I've also never heard parsing being called a pattern recognition problem. Nor is it a machine learning problem; of course grammar induction is, and the two are strongly related, but the parsing phase itself is more like pattern matching than pattern recognition. QVVERTYVS (hm?) 16:58, 5 May 2014 (UTC)

While I also do think "pattern recognition" is a bit broader than machine learning (in the sense of making more use of unsupervised methods - "recognizing patterns" is less restricted to previously known patterns; on the other hand, pattern recognition is often more associated with images as in CVPR), I consider this article to be FUBAR. Let's merge contents worth rescuing into appropriate articles, and then redirect it to Machine learning (or: carefully disambiguate links to machine learning, then redirect Patern recognition to Pattern recognition (disambiguation)). 94.216.95.56 (talk) 17:48, 6 May 2014 (UTC)

You seem to be thinking of "machine learning" as supervised learning only, which is how many people outside the field think of it. But if you look at the JMLR or NIPS, you'll find that they also carry publications about clustering, matrix factorization, and other unsupervised tasks. QVVERTYVS (hm?) 19:34, 6 May 2014 (UTC)

I do, and I have the impression that there are typical patterns to the unsupervised approaches published there, such as minimizing an objective function; that play a much smaller role in database-oriented communities such as KDD. KDD also carries a lot of supervised publications; they aren't disjoint. But there are patterns more common in one than the other. Microsoft academic for example also distinguishes between data mining on one hand, and has ML+PR listed as a separate subdomain. We need to explain why such a distinction is often made, and also why it isn't "just" supervised vs. unsupervised; as both subdomains have both. As Bishop noted, PR seems to come from an engineering point of view (e.g. WP:WikiProject Robotics); ML from the AI background, and data mining/KDD from a database perspective. Not sure where statistics people feel most at home... Maybe this background/focus/coloring is the main difference of these approaches. 94.216.95.56 (talk) 19:55, 6 May 2014 (UTC)

I saw your new intro and it makes me want to merge with the machine learning article even more strongly. Let's leave data mining out of the equation for now.

As for statistics, there's a subfield called "statistical learning" that borrows techniques from ML and tries to give them a firm, statistical basis; Hastie and Tibshirani are the prime exponents, and boosting comes from this field. But generally, as Hinton summarized the situation, statisticians are more concerned with getting signal from small and noisy datasets, while ML people are more concerned with generalizing and squeezing performance out of large, clean datasets (i.e. robustness vs. scalability). QVVERTYVS (hm?) 11:19, 7 May 2014 (UTC)

Given this edit, I think Kri may have an opinion on the matter. In any case, the example of Q-learning got me browsing literature and I found out that Sutton and Barto (1998) make the following distinction:

Reinforcement learning is very diff�erent from supervised learning, the kind of learning studied in almost all current research in machine learning, statistical pattern recognition, and arti�ficial neural networks.

[...] In the 1960 and 1970s, reinforcement learning was gradually overshadowed and lost as a distinct topic, while supervised learning, particularly in the form of pattern recognition, became widely studied.

Unlike in the picture painted by 94.216.95.56, Sutton and Barto consider pattern recognition to be a specific kind (application?) of machine learning, as supervised as the rest of the field. These are also the only two mentions of "pattern recognition" in the document. QVVERTYVS (hm?) 09:24, 16 October 2014 (UTC)

Interestingly, I did find a reference that corroborates the definition given above: Burges, in "A tutorial on support vector machines for pattern recognition" (1998) says "The tutorial dwells entirely on the pattern recognition problem. Many of the ideas there carry directly over to the cases of regression estimation and linear operator inversion, but space constraints precluded the exploration of these topics here." The rest of the tutorial is about classification. Maybe we should merge this article with statistical classification instead? QVVERTYVS (hm?) 21:39, 23 October 2014 (UTC)

Proposed merge with Machine learning

Pattern recognition covers machine learning topics; machine learning covers pattern recognition topics; and I haven't found any source that can really tell the two apart (except that "pattern recognition" is ambiguous and can mean the functioning of part of the brain as well, but that's not what the article pattern recognition is about). I suggest we merge the two, preferably to machine learning since that seems to be the modern name of the field. QVVERTYVS (hm?) 16:55, 21 May 2014 (UTC)

See the section #Disputed, above, for a source that discusses the (non-)difference between PR and ML (a book called Pattern Recognition and Machine Learning, appropriately). QVVERTYVS (hm?) 16:57, 21 May 2014 (UTC)

• Do Not Merge: (1) Learning a game (eg, in TD-gammon) is not pattern recognition, but it is machine learning. (2) A system that recognizes a pattern may be programmed, and therefore need not involve machine learning. (3) Therefore, machine learning and pattern recognition are different, and neither is a subset of the other, though their various relationships and overlaps are important to discuss.

Conclusion: it would be inappropriate to merge merge machine learning and pattern recognition. 86.149.137.127 (talk) 18:40, 19 October 2014 (UTC)

At (2) you seem to be thinking of pattern matching, not recognition. Is learning to see patterns in a game not pattern recognition?

More importantly, do you have any sources to back up your opinion? QVVERTYVS (hm?) 22:06, 19 October 2014 (UTC)

In an edit summary, I gave the example of data compression software, which uses a kind of pattern recognition (as is stated in the article). But does it use machine learning? I guess one could argue that it learns the patterns to recognize while it is compressing a file. On the other hand its performance is reset the next time it is given another file to compress since it forgets all the patterns it has learned, so it would be a very special kind of machine learning. But maybe it is as you say that pattern recognition is a form of pattern matching where the patterns are learned rather than preprogrammed. —Kri (talk) 20:40, 20 October 2014 (UTC)

86.149.137.127: Agreed—not all systems that utilize machine learning are capable of pattern recognition. I would say that many of them can do generalization, though, but that is a different thing. —Kri (talk) 20:55, 20 October 2014 (UTC)

• Support merge: Even though the words themselves may give rise to different associations, machine learning and pattern recognition as research fields are basically synonymous today. The names are not really transparent in this sense, rather they are more like set phrases that are applied to a research field that at the same time does more and less than what the name would imply. The two names reflect the two historical origins. Pattern recognition comes from engineering and machine learning comes from computer science. But they are pretty much united today. Qorilla (talk) 00:31, 21 October 2014 (UTC)

Just a question, would you consider reinforcement learning, decision tree learning or genetic algorithms to be a form of pattern recognition? —Kri (talk) 14:20, 21 October 2014 (UTC)

Kri, I personally wouldn't consider GAs to be machine learning either (as I've argued elsewhere) except by historical association. DTs are considered part of PR at least in Scholarpedia's categorization. Reinforcement, in the psychological sense, is also listed there, though Reinforcement learning is not.

Interestingly, while Scholarpedia has separate categories on Machine learning and Pattern recognition, the two are combined into "Machine learning and pattern recognition" in the Computational intelligence overview. QVVERTYVS (hm?) 10:28, 13 November 2014 (UTC)

I don't think GAs are machine learning either anymore, not any more than gradient descent since they are both optimization algorithms, which I think we all could agree on. I wouldn't consider an artificial neural network alone to be machine learning either. I think machine learning is when a machine actually learns something. And to make it learn, GAs, GD and ANNs can all be used, so they can be used in machine learning, but they are not machine learning because of that. So maybe I used the wrong term.

About decision tree learning, yes, it can most likely be used to recognize patterns, while reinforcement learning is probably almost never used to do that, even though I can imagine that this also could be used in some way to detect patterns. However, all decision tree learning and reinforcement learning are forms of machine learning. What I wanted to say I guess was that I think pattern recognition is a subfield of machine learning. —Kri (talk) 12:50, 13 November 2014 (UTC)

But then you would limit machine learning to a set of algorithms. Neural networks as models are also a subject of study in the field of machine learning (see, e.g., Learning Deep Architectures for AI in Foundations and Trends in Machine Learning, which discusses models as well as algorithms). I remember Hinton saying somewhere in his Coursera course that at one point, machine learning and neural nets were considered disjoint fields with ML journals not accepting NN papers, but they have since merged back together. QVVERTYVS (hm?) 13:04, 13 November 2014 (UTC)

I don't consider machine learning to be any set algorithms, I consider it to be when machines learn, as I stated it. GAs are also a subject of study in machine learning. —Kri (talk) 13:29, 13 November 2014 (UTC)

Or well, the field of study of machines learning. —Kri (talk) 13:36, 13 November 2014 (UTC)

• Neutral: Even if we do merge, machine learning and pattern recognition will still not be the same thing (as implied by their names), and it is important to make a distinction between the two in order for pattern recognition not to get blurred out and become equivalent to all forms of machine learning. I therefore think that pattern recognition should be a separate section in the machine learning article, if the discussion would result in a merge. —Kri (talk) 14:16, 21 October 2014 (UTC)

However, I think this also depends on what you mean by a pattern. But for most people I guess a pattern in something that keeps on repeating, in which case pattern recognition would be the recognition of something that keeps on occurring again and again. In this notion, the repetition also has to be somewhat predictive, since for example in a random sequence of ones and zeros there is no pattern to be recognized (except from all numbers being ones and zeros), but "1" keeps on repeating, although not in a predictive way. —Kri (talk) 12:33, 13 November 2014 (UTC)

• Do not merge:

  There may be significant overlap between the fields, but they are not identical. Are there any pattern recognition textbooks that cover Inductive logic programming and Genetic Programming which are concerned with learning programs written in Prolog and Lisp?

  Perhaps learning a classifier belongs to pattern recognition but learning a program does not? pgr94 (talk) 21:16, 7 November 2014 (UTC)

No, but then machine learning textbooks don't often cover those topics (anymore) either. Mitchell's 1998 textbook did, but Bishop's 2006 one does not, nor does The Elements of Statistical Learning (which is arguably biased towards traditional statistical methods, but also covers neural nets). Judging from the titles of accepted papers at this year's ICML, no-one there was talking about learning Prolog or Lisp programs. QVVERTYVS (hm?) 22:14, 7 November 2014 (UTC)

2014: 24th annual conference on ILP, sponsored by the AI journal and ML journal. There was special issue on ILP in the Machine Learning journal in Jan 2014[2] and there's another special issue with a deadline next month. A glance at google scholar will show this field is healthy and active[3]. You didn't look very hard, Qwertyus. pgr94 (talk) 23:03, 7 November 2014 (UTC)

Fair enough, that shows ML people still consider ILP to be part of their field and it's indeed still alive. I guess I am biased by my environment.

Anyway, my point is not that ML and PR should be the exact same thing, but what is covered by our article about PR is really a bunch of supervised learning methods, without any sign that pattern recognition ≠ supervised learning. QVVERTYVS (hm?) 10:29, 8 November 2014 (UTC)

• Do Not Merge I'm amazed this is even being discussed. They are clearly different topics. Absolutely do not merge. The fact that the current PR article equates it to ML is a problem with that article but the way to fix it is not to compound the problem by merging the articles. --MadScientistX11 (talk) 19:57, 12 November 2014 (UTC)

Ok. Do you any good sources that point out the difference, or some kind of definition of pattern recognition as a problem or field? QVVERTYVS (hm?) 22:16, 12 November 2014 (UTC)

I'll look through some of my books to see if I can find something. But here is an example that I think supports that they are different. One of the first and most famous examples of machine learning had nothing to do with pattern recognition. That was the program of Arthur Samuel that learned to play checkers through trial and error. --MadScientistX11 (talk) 22:54, 12 November 2014 (UTC)

Also, the book Machine Learning: An Artificial Intelligence Approach by Michalski et al. That book is filled with examples of machine learning many of which have little to do with pattern matching. E.g., the use of theorem provers, heuristics, case-based reasoning, Cyc, etc. --MadScientistX11 (talk) 23:02, 12 November 2014 (UTC)

If I may commit some original research: the Samuel program found patterns in boards, the patterns of "good" boards. IIRC, the pattern was a linear combination of the pieces on the boards, with own pieces having value 1 and opponent pieces -1. (To make this less OR, check out the Pattern Recognition category on Scholarpedia, where TD-Gammon is tentatively placed.)

Anyway, neither example is convincing that pattern recognition is not subsumed by machine learning. My point is not to equate them, and that is not even what the current article does: it lists only machine learning techniques, but not all of them. QVVERTYVS (hm?) 00:20, 13 November 2014 (UTC)

You are completely wrong about the Samuels program. It didn't work by finding patterns at all. It worked by successive refinements of the program. There were a bunch of variables that Samuels defined that controlled the way the program worked and based on the results of each game the program would adjust the variables. It had nothing to do with finding patterns. And look at the various names and topics in the book I referenced: Hayes-Roth, Lenat, etc. The kinds of problems those people work on for the most part (expert systems, symbolic reasoning, developing large ontologies) had nothing at all to do with pattern recognition. --MadScientistX11 (talk) 01:01, 13 November 2014 (UTC)

Ah, it seems I'm confusing Samuel's program with something else. However, I'm not convinced that a book from 1986 is a good reference for the state of both fields, and I have this editorial in Machine Learning on my side: "the early volumes of Machine Learning [late 80s] included a variety of papers on problem solving, reasoning, and language, but by the mid-1990s they had nearly disappeared from the literature". The GOFAI approach has been abandoned for a few decades now, and is mostly of historic interest. QVVERTYVS (hm?) 09:42, 13 November 2014 (UTC)

• Do not merge: As has already been discussed previously there is a significant overlap between the two topics. However, there is also a considerable portion that is disjoint. I think it is more appropriate to say that pattern recognition is a branch of artificial intelligence, not machine learning. One can create a pattern recognizer without the use of machine learning (ie: learning a model from the data), simply by dictating the rules. Such rule-based systems can often be improved upon by taking use of machine learning. See Bishop 2006. pp. 1-2. That being said, I'm all for a cleanup. Merge the redundant parts and keep the unique separate. Hvaara (talk) 14:08, 16 November 2014 (UTC)

Ok, I'm withdrawing my suggestion since it's met too much opposition. QVVERTYVS (hm?) 19:02, 16 November 2014 (UTC)

This article has been renamed Pattern recognition (machine learning)

This article's title has been disambiguated in order to prevent it from being confused with Pattern recognition (psychology). A part of this article's lead section conflates these two meanings of "pattern recognition". I hope the article's new title will help to prevent the misunderstandings that led to this article's factual inaccuracy. Jarble (talk)

I have reverted. Please use WP:RM since I suspect it is not uncontroversial and your move resulted in a WP:MALPLACED mess which broke hundreds of inbound links and several high traffic redirects. —Xezbeth (talk) 10:22, 1 May 2015 (UTC)

Why is this labelled machine learning?

Why is Pattern Recognition labelled as machine learning? That's similar to saying that Neural Networks are machine learning. ANN are usually generated through ML, PR models are often generated though ML, but they're both not ML...

The reference in the very first paragraph points to a PDF entitled "Pattern Recognition *and* Machine Learning".

PR may go hand in hand with ML, just like ANNs do too. They're not ML by themselves, though.

.. no? — Preceding unsigned comment added by 2.219.220.41 (talk) 00:42, 19 April 2018 (UTC)