Talk:Analytic hierarchy process

This is the talk page for discussing improvements to the Analytic hierarchy process article. This is not a forum for general discussion of the article's subject.

Put new text under old text. Click here to start a new topic. New to Wikipedia? Welcome! Learn to edit; get help. Assume good faith Be polite and avoid personal attacks Be welcoming to newcomers Seek dispute resolution if needed Article policies Neutral point of view No original research Verifiability

Find sources: Google (books · news · scholar · free images · WP refs) · FENS · JSTOR · TWL

Archives: 1, 2, 3, 4, 5, 6

Archives

Archive 1 - 03/05 through 08/07

There was an unsigned change that reintroduced some very old content in this article and it previously caused a lot of problems. I changed it back to the version last edited by DCLawyer. I have a feeling this will be another slow-paced edite war so let me just state the problems with this proposed version of the article. 1) It makes an ad hominem argument by simply saying that the criticisms are a lack of understanding of AHP by the critics. Example: "AHP has had challenges to its theoretical and practical shortcomings from those who may not fully understand it." and "This criticism illustrates a lack of understanding for several reasons." The critics are numerous and the rank reversal problem has attracted quite a lot of attention. A review of the citations show that the critics are quite qualified and the criticisms are based on a mathematical analysis of the problem. I think, however, it shows a lack of understanding of the mathematical level of the issue by the editor who insists this is the correct version 2) This editor is again making the claim that AHP could be used to "solve" problems that already have formally proven solutions. Curiously, the issue of whether proponents of AHP claimed it could be used to solve problems such as actuarial calculations and optimal wing designs came up earlier in this discussion page. The article previously pointed out that proponents actually do make such claims. But, it was concluded that it could not be stated in the article that proponents make such claims because this information was only based on the claims made in previous version of this article. And yet here, againm, this editor is saying exactly that. It seems that the burden should be on this editor to show that AHP has been used on such problems and that it actually produces the same answer as the known mathematical optimization methods. 3) The language gets very flowery and POV. Example: "One of the amazing things about the AHP..." 4) I suspect this is a COI problem. The Saaty's have previously been warned about making these changes themselves. This could also be one of the software vendors that keep posting their products here.

The editor (if he/she is the same as the one who made these edits before) who keeps reintroducing this material refuses to even discuss it on the talk page even though they have been encouraged to discuss these issues in detail. Again, I make the offer to discuss the criticisms of AHP here in the discussion page.Hubbardaie 23:11, 3 September 2007 (UTC)[reply]

In my opinion, you were right to revert the edits. The anonymous person who made them didn't provide any sources for his/her claims (except for maybe one, IIRC). He/she may not know that Wikipedia requires sources for all claims, especially those that are subject to challenge or question.

For example, if you're gonna say that AHP has been challenged, you need to cite one or more reliable sources where such a challenge has been made. Otherwise, people will remove your claim. For subjects like AHP, books or peer reviewed journals are the best reliable source. Somebody's web site, probably not.

The same goes for claims that AHP is wonderful, or amazing, or better than sliced bread, or whatever. To say that stuff yourself is original research, and is not allowed. To quote or accurately paraphrase a reliable source that said it is acceptable, provided it is done in an evenhanded way. ("A study at the University of Foo found that shoppers preferred AHP to sliced bread by three to one," followed by a citation of the study's publication, preferably in a non-U of Foo place, would probably be OK.)

It would also be helpful if this anonymous editor got an account so people could discuss these things with him/her on their talk page. DCLawyer 23:45, 3 September 2007 (UTC)[reply]

I deleted a paragraph that had some weasel words and a citation request that had not been acted on after more than two weeks. Ohio Mailman 23:42, 9 September 2007 (UTC)[reply]

I deleted a sentence where no citation had been provided over a month after it was requested, plus a sentence that depended on the first one. Ohio Mailman 22:38, 30 September 2007 (UTC)[reply]

I removed the material about Egon Brunswik's Lens model. The reference provided to support it has nothing to do with scoring methods or the track record of Brunswik's model vs. the Analytic Hierarchy Process. Nothing else in the Kleindorfer book supports the claim of other methods being superior to AHP, or even discusses a comparison. The book contains only one brief reference to AHP, on page 224. It is a favorable mention, with the context that AHP "...enhance(s) the information-processing resources available to the group through the intelligent use of information technology." Ohio Mailman 15:01, 4 October 2007 (UTC)[reply]

I've added this section and wikified it. I hope everybody likes it. I'm working on some additional material to describe AHP more fully. Most of it is available for online examination and comment. If you want to look at it, let me know. Lou Sander 21:02, 23 October 2007 (UTC)[reply]

The first sentence of the "uses and applications" section gives BhusBhushan, Navneet and Kanwal Rai book as a source of the claim about what AHP is "best" used for. But I reviewed this source and while the book does make such assertions, I see no emprical evidence in the book supporting this claim. Of course, I'm not including examples of where it was used and where people were pleased with the results as evidence that deicisons were improved. I think all once can fairly claim from this source is that "proponents believe...". I added those two words. But if there is some other explicit and conclusive empirical basis for these claims then that source should be used and the "proponents believe..." qualification can be removed.Hubbardaie 02:19, 28 October 2007 (UTC)[reply]

I'm going to go ahead and to remove the words "Proponents believe" from this section. They cast doubt on a well-sourced fact that is extensively supported by other literature on AHP. It's not up to us editors to analyze the content of verifiable sources, or to include our personal research into their validity. 136.142.73.116 15:44, 30 October 2007 (UTC)[reply]

I agree with your edit, but I kind of wish you'd get a user name. This article has had its share of bad edits by anonymous users. DCLawyer 00:45, 31 October 2007 (UTC)[reply]

There are many "well-sourced" claims that are nonetheless controversial and not at all universally held by specialists in the field. If there are sources of specific empirical evidence of this claim and not just the claims of an AHP evangalist, then those sources should be added before the "proponents claim" qualifier is removed. I'll put it back. Hubbardaie 17:47, 5 November 2007 (UTC)[reply]

The anon is right. I reverted the original research. Cleome 08:03, 6 November 2007 (UTC)[reply]

I realize you believe the anon to be correct. The "original research" would be not to include this qualifier since "proponents believe" is all that one can derive from the source given. Since the authors of the source mentioned no substantial empirical evidence (only anecdotal cases), the original research would be to claim more than can honestly be claimed from that source. In the spirit of avoiding edit wars, I'll wait a few days for a response before I revert back to the proposed qualifier. Perhaps someone can source some published research that can justify the removal of "proponents believe". Hubbardaie 21:25, 6 November 2007 (UTC)[reply]

By the way, I'm getting the impression that I'm the only one who has read the single source provided for this claim. I don't think anyone who read it would call it "well sourced" since the source provides no evidence for the claim. I encourage those who want to insist the the claim is well sourced to read the source and point to the specific page where empirical evidence (not a mere case study) of the claim is provided. I'll wait a couple more days to revert to what I wrote just in case someone can come forth with this request. Hubbardaie 13:25, 7 November 2007 (UTC)[reply]

It seems like you are trying to twist or analyze the content of a reliable source. It isn't up to us to do that. See WP:V, which says "The threshold for inclusion in Wikipedia is verifiability, not truth. "Verifiable" in this context means that any reader should be able to check that material added to Wikipedia has already been published by a reliable source." Good Cop 08:32, 8 November 2007 (UTC)[reply]

No, no. You have it exactly backwards. To say that "the authors/proponents say X" is the undisputed fact among us. To presume that the authors have proven X by asserting X to be true without qualification, is the attempt to "twist or analyze the content" of the source. Yes, it is verifiable that these authors say that - and to say any more than that is editorializing. But I tell you what I'll do...I'll add other verifiable sources that claim exactly the opposite. Then we'll have to deal with how to combine contradictory claims in the article. Of course (like many other articles), we will find that the best way to combine these is through the acknowlegement of the controversy. More to come. By the way, how many "verifiable sources" do you think there are for the claim "astrology works" if we don't worry at all about the other side of the argument in that claim? Hubbardaie 12:59, 8 November 2007 (UTC)[reply]

I added another source and another point in criticisms. This is a respected peer-reviewed journal, not a single case study written by proponents. I also removed a claim that was half uncsourced (the part that said the criticisms are "mostly theoretical or speculative" and half redundant (the same case was already and more appropriately listed in applications). To assert that the criticims are merely "theoretical or speculative" in the face of these sources would, itself, be editorializing. Now, before anyone removes or qualifies the comment further, remember I'm using the same rule used by Good Cop above. The source only needs to be verifiable and whether there is controversy is irrelevant. According to Good Cop, as long as some verifiable sources states it, it can simply be asserted without further qualification. Hubbardaie 13:33, 8 November 2007 (UTC)[reply]

Skyrocket, after a well-thought-out pause, managed to muster "please just leave it alone" as the sum total of his/her response. Try to scrape up something a bit more articulate. We are trying to construct an ecyclopedic article, here. Think through the response and respond to specific comments I made. I know you can do it. Hubbardaie 06:43, 10 November 2007 (UTC)[reply]

I've replaced the old lead with a new one and wikified it. Points that were in the old lead but not this one will be added to subsequent sections. Lou Sander 16:40, 24 October 2007 (UTC)[reply]

I replaced the above "new" wikified lead with a newer one that is more complete and accurate, and probably better written. Lou Sander 02:53, 8 November 2007 (UTC)[reply]

I reverted the removal of properly-sourced material. The reverted edit also altered the meaning of the paragraph and included redundant mention of the ten year old Schenkerman article. Ohio Mailman 20:52, 8 November 2007 (UT

No the source does not support the first part of the claim that the flaws are "mostly theorectical or speculative". It is only a source that it was applied in the case specified, and that case is already cited in the applications and uses. The mention of one particular case study simply doesnt belong in this section, anyway. And regarding the claim that the flaws are "mostly theoretical or speculative" you will have to provide a source specifically for that claim (which is not the source you provided). The age of the Schenkerman article is only relevant if you can cite a specific article that addresses Schenkerman's objections. I've researched this and I doubt you will find one. I think what you will find is that other simply drive on and generate new applications of AHP without ever addressing (or probably even being aware of) the earlier objections. Finally, I list three other critics besides Schenkerman. I could have easily added many more from peer reviewed journals (not merely a self-promoting book written by AHP proponents, like the source you quote). You will need to specifically cite a source that supports the claim that the criticisms are mostly theoretical or speculative before you should re-insert it. Hubbardaie 21:55, 8 November 2007 (UTC)[reply]

Also, if you insist that any "properly sourced" material cannot be removed, then you must apply that same standard to the Schenkerman citation in the first paragraph. I think the age is less relevant than the fact that the Schenkerman article is in a respected peer reviewed journal and the source you reinserted is not. But that dispute is irrelevant if the criterion for leaving in a claim is simply that it be "properly sourced". You cannot remove this citation and insist the other cannot be removed. It's been made clear in here that parsing a citation any further than that it exists is editorilizing and OR, so we should apply the same rules to all the citations. Hubbardaie 22:25, 8 November 2007 (UTC)[reply]

I have no problem with that source, but its already mentioned in the applications & uses and I'm not sure why that one alone (a watershed mgt case) is the one citation that should be used to make the point that it is popular. Seems like an one arbitrary choice out of the list of case studies. The applications & uses section already discusses its popularity. The "despite these concerns..." sentence also sounded partially redundant with the first sentence in the paragraph. Since the first sentence already mentions it is widespread, I added that it is popular - so that should cover both points adequately enough for a section that is supposed to be about criticisms. If we think we need to add such qualifying statements to the criticisms section, then it should make sense to add similar qualifiers to the (very flowery editorial) introduction. For example, we should introduce it as the "popular but controversial" method. I just think that a balanced article would either mention contradictory views in both the introduction and criticisms sections, or neither. If you insist that the sentence should be reinserted, then I will also add the "controversial" qualification in the introduction to balance out both sections. Hubbardaie 02:19, 9 November 2007 (UTC)[reply]

Someone keeps removing the properly-sourced sentence "The resulting decision errors cause AHP users to be "seriously misled".[17]". When I previously attempted to remove a claim I felt was editorializing, others insisted that it was "well sourced" and should not be debated as long as source can be verified. Now I have a well sourced claim (unlike the other source, this is a respected peer reviewed journal) and it keeps being removed. We need to have some consistency here. If this claim can be removed, then so can others, regardless of whether some think it is well-sourced. Hubbardaie 03:02, 9 November 2007 (UTC)[reply]

Good Cop has deleted a properly sourced statement in a respected peer-reviewed journal. He has now contradicted his previous position that as long as a statement is well-sourced it is editorializing to remove it. He is applying different rules for statements he agrees than the statements he disagrees with. The source of the statement he chooses to replace this one with, however, is not a peer reviewed journal and on that basis alone it should not trump a peer reviewed journal. If he insists that it is false, contrary to a citation of a respected journal, then that is OR.Hubbardaie 04:43, 9 November 2007 (UTC)[reply]

Good Cop insists I'm 1) distorting the meaning of the first paragraph and 2) distorting what Schenkerman said. First, I would like to hear why Good Cop thinks he has a special authority to decide the the meaning of paragrapsh should be. This is the criticisms section, and the first paragraph should introduce the criticisms - not just reiterate the flowery, non-judgemental language of previous sections. Secondly, Schenkerman's entire quote is provided a few lines later for all to see. Any reasonalbe reviewer would have to conclude that I provided an exact quote and the tone is entirely consistent with the tone of the entire quote in context. Good Cop previously insisted that well-sourced statements must not be removed and to do so was editorializing. He is now doing exactly that. Clearly, Good Cop thinks that "balanced" is to offer no challanges to the claims of AHP in the introduction but to offer challenges to the challenges in the criticisms section. Here is what I propose. I will add the qualifiers to the introduction section that the method is controversial. Let's see if Good Cop will apply the same rules to both sections. Hubbardaie 12:25, 9 November 2007 (UTC)[reply]

Good Cop is right. Ohio Mailman 14:38, 9 November 2007 (UTC)[reply]

You think Good Cop is right about him having special authority to decide the meanings of paragraphs? Or do you think he is right about me twisting Schenkerman? Or is it something else you haven't articulated? If the second one, see my detailed response below.Hubbardaie 23:18, 9 November 2007 (UTC)[reply]

I strongly agree with Ohio Mailman. Skyrocket654 16:24, 9 November 2007 (UTC)[reply]

Read what I wrote to Ohio Mailman and what I wrote below. You might not understand what you are strongly agreeing with.Hubbardaie 23:18, 9 November 2007 (UTC)[reply]

"Strong agreements" aside ("strongly [more important]" is 5 points on the AHP scale, so that's a lot, right?), Schenkerman's quote did not specify only situations where the erros occur so, of course, he would say "users can be seriously mislead". I made it clear that I was already specifically talking about those situations where this errors occured in which case, the users WILL be mislead (by definition of "error"). But since you found it confusing I replaced it. Again, this is the criticisms section. One cannot honestly read the criticisms sourced as then continue to presume that AHP is "proven" to work and make unchallenged, flowery claims about it in the rest of the article. That's why an honest and truly evenhanded approach would be to point this out. I moved the watershed management case citation to a "Proponent Responses" sub-section. I still don't know why you think that particular case study is the necessary and sufficient evidence of that point, but you may leave there if you like. For me, that was more of a point of basic logical article construction rather than a problem with that particular citation. On another note, if Good Cop would have specified his problem with "can be" vs. "will" in the first place (he mentioned that on my talk page, not in this discussion for some reason) instead of just deleting the whole sentence then we could have made a simple change and saved us both some time. I also suggest Good Cop review the WP:AGF (I’ve told him about this before in his behavior), WP:3RR and WP:OWN (he complained if me “destroying the work of other people” in a section I mostly wrote). Hubbardaie 23:13, 9 November 2007 (UTC)[reply]

i agree with hubbardaie Sweet Death (talk) 01:50, 7 February 2008 (UTC)[reply]

I just corrected a claim in the first paragraph of the criticisms section that said that it was the use of arbitrary scales that caused the "rank reversal" problem. The problem of rank reversal is not a result of that issue and the reason arbitrary scales are a problem is not related to rank reversal. I simply deleted the reference to rank reversal (for now).ERosa (talk) 17:08, 16 February 2008 (UTC)[reply]

I've added some material about AHP hierarchies. Lou Sander 18:43, 14 November 2007 (UTC)[reply]

Most of the initial AHP development was in the 1970s, not the 1980s. The seeds of it were sown from 1963-1969 when Dr. Saaty worked for the State Department (See his "Fundamentals of Decision Making and Priority Theory," page ix.) The first book on it was published in 1980 (Saaty's "The Analytic Hierarchy Process: Planning, Priority Setting, Resource Allocation.") Lou Sander (talk) 15:09, 3 February 2008 (UTC)[reply]

what i deleted was irrelevent. article is imbalanced. Sweet Death (talk) 01:48, 7 February 2008 (UTC)[reply]

There are some criticisms of this analytical process, but they have not found wide support. This section does not present the situation fairly --

• Undue weight is given to 18 year old debates among professors, including extensive quoting from one side.
• The claim that "the use of arbitrary scales and certain internal inconsistencies and theoretical flaws have been discussed extensively in the literature" is supported by only two (2) citations, both over ten (10) years old. Nobody in this century ever pays attention to them, let alone repeats them.
• The use of Wikipedia subheadings gives these matters undue importance in the article's Table Of Contents. They are of historical interest only.

How can these problems be fixed? Skyrocket654 (talk) 18:59, 10 February 2008 (UTC)[reply]

It seems Skyrocket654 has a point here. Because I am rather new at being a Wikipedia editor I am reluctant to make any changes myself.MathDame (talk) 17:03, 11 February 2008 (UTC)[reply]

I have also been bothered by the many TOC entries for this small section. I am fixing the problem by replacing the headings with bolded letters. Hope it's O.K. Cleome (talk) 18:38, 11 February 2008 (UTC)[reply]

I noted ERosa's remarks. 1) If all we have is 18-year-old stuff, we shouldn't say a lot about it. Maybe mention that there were debates 18 years ago, and provide the references. 2) If the critics are still arguing that their charges weren't answered, we should include current references. 3) We shouldn't care very much about proponents or opponents or what they might think, or how many there are, and so forth. Let them publish their stuff. We can include it if it's appropriate to do so. DCLawyer (talk) 02:06, 12 February 2008 (UTC)[reply]

I have been bothered for months by the quotations from those long ago debates. There were at least eight or nine articles in the series. It is not our job to summarize them, especially by arbitrarily selecting quotations. But the debates did take place, and there is no reason not to mention them. Readers who are interested in them can get a complete picture by reading the articles in the references.

The references should be kept but the quotations should be deleted. All this can be fixed by deleting everything between "In these debates,..." and "...good basis from which to develop." The two references that appear there are duplicates of those in the "A series of debates..." sentence. Ohio Mailman (talk) 16:17, 13 February 2008 (UTC)[reply]

Again, note my remarks below about why the age of a still-valid criticism matters not. The question should only be about whether they represent the weight of current academic thinking (people tend not to republish the same criticism even though it is still valid since anyone can just cite the original). Some here have remarked that these criticisms "have not found wide support" but if that is the case they should cite their sources. I find these quotes to be the predominent thinking among people in the decision sciences but, again, a claim either way would be unfounded and OR without a verfiable source.ERosa (talk) 23:18, 13 February 2008 (UTC)[reply]

But I thin Ohio Mailman makes a valid point about the quotes. All criticisms should stand (rank reversal should be re-entered)but could be replaced with a description of the debating positions instead of quotes.ERosa (talk) 23:20, 13 February 2008 (UTC)[reply]

There are some famous theorems in geometry that are over 2500 years old, and they are still good. The problem is that the criticisms were sound 18 years ago and the proponents never answered them in a way that is conclusive proof that the initial criticism is wrong. Contrary to what DCLawyer said, the burden of proof should be on the side that has not provided citations for how the critics were answered. If these criticisms are "out of date" then provide the citation that conclusively disproved them. Otherwise, assume they are still valid. To simply label them old without specific citation amounts to original research. Let's avoid breaking the NOR rule. I could point out that the criticisms of areas such as phrenology, astrology, the ptolemaic system, and certain aspects of Aristotelian physics are also quite old. If we use DCLawyers logic, should we now conclude that the criticisms have worn off and it is the burden of the critics to cite new criticisms? No,until the old ones are answered conclusively, they are still good.ERosa (talk) 02:43, 12 February 2008 (UTC)[reply]

The debates started 18 years ago but the critics argue that their charges were never adequately answered. No empirical findings to date have overturned these criticsms. If anyone has information to the contrary, please cite the specific source of the controlled experiment (not just testimonials from a case study) or the mathematical proof of the error in the criticism. I realize AHP proponents are almost religious in their support of AHP and prefer not to admit to any current debate on the topic. But the debate is real and is still real after 18 years. Furthermore, you should compare the relative emphasis given to criticisms sections in other methodologies an I think you will find that this is entirely consitent with how criticisms are presented in peer-reviewed literature. Finally, I'm not sure what Sweet Death meant by not having wide support, but among academics I find very few pro-AHP researchers. The one exception I found as a student of Saaty who started an AHP software company. On the other hand, if you feel that you have empirical evidence of how wide or narrow support for AHP is among objective researchers (not, of course, among AHP vendors and managers who bet their careers on buying the AHP tools) then you should cite that in the article. ERosa (talk) 18:56, 11 February 2008 (UTC)[reply]

I also just noticed that the history of the article included a point about rank reversal. Proponents argue that this has been addressed with the "ideal process mode". But this is still of interest at least historically because it was a key criticism for many years. Its also interesting that Saaty's initial argument was that when rank reversal happens, it actually *should* be happening and is not, as the critics charges, evidence of a fundamental logical flaw in the process. It was only much later that Saaty apparently agreed that rank reversals are often (I would say always) an absurd result and then developed the Ideal Process mode to correct it. My concern is that when a method has such a fundamental flaw, one has to be concerned about the entire approach and that just fixing the outcome after the fact is putting lipstick on a pig. But, since I don't want to be accused of original research, I'll see if some other source has already responded to Saaty's fix to rank reversal and write a section that cites it. In the mean time, I think rank reversal is important enough to at least be mentioned as part of the historical evolution of AHP.ERosa (talk) 21:15, 11 February 2008 (UTC)[reply]

I decided to be bold and make my first edit to this article. I found some good accurate words in two of the existing references and I thought they were better than what was there. MathDame (talk) 03:14, 15 February 2008 (UTC)[reply]

I don't know much about this phenomenon, but it seems like it gives rise to an important criticism of the AHP. Apparently this criticism has been around for a long time, and at least a few people might still be talking about it. We probably should say something about it in the Criticisms section. Here is what I think should be said:

• We should acknowledge that there is a phenomenon called "rank reversal," and that some people think it's a problem.
• We should describe the phenomenon, in a way that is appropriate to a general encyclopedia. Maybe include a practical example.
• Also, in an appropriate way, we should say something about what critics have said about the effect of rank reversal on AHP's suitability, reliability, etc. If there is a debate involved, which it seems there is, we should also say something about the rebuttals to the criticisms. We should NOT cherry pick quotes. Hopefully, somebody can find a reliable, unbiased 21st century source that summarizes the debate.
• We should carefully avoid giving this criticism undue weight. AHP is used, studied, and taught very extensively in many countries of the world, by intelligent people who know what they are doing. Whatever rank reversal is, it does NOT seem to be something that "proves" that AHP is no good, or that seems to be having much visible effect on its use.

I'm not very interested in pursuing this matter myself. I'm just putting in my two cents worth. Skyrocket654 (talk) 15:51, 15 February 2008 (UTC)[reply]

What you've said is fine but I would caution that a method being widely used is no evidence of it actually working or being beneficial. If that were the case, we would have to acknowledge astrology as even more beneficial. The point that some of the critics (including those listed in the article) have made is a lot of people are spending time with a method that - they argue - doesn't really work at all. I think the fact that AHP is widely used is already clear from the article but the criticisms section should not necessarilly be blunted as if wide use were evidence the criticisms are invalid. Regarding this specific criticism, we should mention that Saaty's "Ideal Process Mode" is meant to correct rank reversal. The fact that Saaty originally argued that rank reversal SHOULD occur and then, contrary to his original position, created a method to "fix" the probem may give the issue undue weight as a criticism. But another point I would make is that the issue of rank reversal doesn't even necessarilly have to be a criticism. It is simply part of the evolution of AHP. How a method evolved does seem to be a legitimate topic in articles about other methods.ERosa (talk) 17:56, 15 February 2008 (UTC)[reply]

We are not dealing with astrology here. AHP is being used and taught by large numbers of well-qualified and well-informed people. If the critics are arguing that the method doesn't work and that people are wasting their time with it, a lot of serious people don't seem to accept their arguments. DCLawyer (talk) 09:55, 16 February 2008 (UTC)[reply]

But you need to understand that the critics are also well-qualified and well-informed. In fact, I would argue that the vast majority of AHP proponents would nor could never be published in the same level of journals as the critics cited. And, yes, if the critics are right then a lot of people *are* wasting their time with it. The mathematical flaws the critics refer to were not arrived at by popular vote. It wouldn't be the first time in history that a popular method (even one with some academic proponents)was debunked. And the popularity of it is all the more reason to point out the flaws (by the way, astrology has also had academic support in the past - so they are, in fact, identical situations).ERosa (talk) 13:59, 16 February 2008 (UTC)[reply]

By the way, DCLawyer should review the Wikipedia article on the fallacy Argumentum ad populum. I initially assumed a trained lawyer would be wary of this common fallacy but I guess not.ERosa (talk) 14:13, 16 February 2008 (UTC)[reply]

I just changed a version that claimed there was a "...the general consensus that the [AHP] technique is both technically valid and practically useful,..".  The cited source was merely an online journal for software developers, not actually an academic research journal in the decision sciences.  If anyone has a proper source for such a claim, it should be cited instead of this one.  However, you will not find any such claim in a peer reviewed journal.^{[citation needed]}  People who know the controversy behind this topic wouldn't even claim AHP proponents are even the majority among researchers, much less approaching "consensus".^{[citation needed]}  I suspect anyone who thinks there is a consensus is not a researcher but probably one of the vendors or customers of AHP, who are mostly unaware of the problems in AHP or, at least, don't seek out the numerous critics.

The cited source was MUCH less reputable than the sources cited for the criticisms.^{[citation needed]}  I would recommend, in fact, that if anyone wants to claim there is anything remotely like a "consensus" that AHP is technically valid, they should cite a source as reputable as those cited for the criticisms (Managment Science and the Journal of the Operations Research Society). Be advised that in the decision sciences, that's about as high a standard as there is.

I did change another point.  The previous version of the criticisms claimed that it was the use of arbitrary scales that resulted in "rank reversal".  Actually, that is not the reason rank reversal occurs.  It occurs because of the nature of matrices based on pairwise comparisons both within and among the columns.ERosa (talk) 04:54, 16 February 2008 (UTC)[reply]

ERosa: We realize you are new here, but please do not remove properly sourced material from Wikipedia. And please do not corrupt the formatting of the article. Using the Show Preview button will avoid this. If you need help in editing, others will usually be happy to provide it. DCLawyer (talk) 09:22, 16 February 2008 (UTC)[reply]

And I realize you are probably new to research in this area (I'm not) but it was not properly sourced. It was an off-hand comment in an online journal artile for software developers and the author provided no evidence whatsoever of this claim. In fact, the citation of rank reversal that was in here over a month ago WAS properly sourced and was removed. If you insist on leaving in such a weak source from someone trained in an unrelated topic, then the comment needs to make that clear. You need to think very carefully about whether any standards should apply at all to sources or it would be possible to cite a source for virtually anything.ERosa (talk) 14:04, 16 February 2008 (UTC)[reply]

Also, I apologize for the formatting problem. I'll use preview as you suggest.ERosa (talk) 14:24, 16 February 2008 (UTC)[reply]

ERosa: Please do not change the clear meaning of properly sourced material. It is not up to editors to interpret or evaluate it. Articles may not contain any new analysis or synthesis of published material that serves to advance a position not clearly advanced by the sources. (See WP:OR). Also please refrain from making comments about other editors. Please do not give them unsolicited advice. Comment on content, not on the contributor. These things can be seen as personal attacks. (See WP:NPA) DCLawyer (talk) 15:08, 16 February 2008 (UTC)[reply]

Regarding your "citation needed" comments within this talk page, please do not confuse discussions with actual edits to the article. If citations were needed on each sentence in the talk page, I would certainly pepper yours. Also you apparently see no irony in your statement "Please do not give [authors] unsolicited advice". Have I solicited any from you? I will change it again in a way that attempts to satisfy your need to position a quote from a little-known online journal for software developers to equal standing with the peer-reviewed Management Science.ERosa (talk) 16:25, 16 February 2008 (UTC)[reply]

Without removing it, I put the "general consensus" claim in its proper context and moved another properly sourced quote next to it to show the contrast. Since my comment simply explains the source, it cannot be deemed as "changing the meaning" of it in any way. Any change to the part I just added should meet the standards as stated by DCLawyer as 1) we should not change the meaning of the properly sourced quote (it was previously presentd as if it were an uncontested fact, not simply a comment in an online journal unrelated to the decision sciences) 2) we should not remove or change the meaning of the properly sourced quotes from the critics, 3) there should be no original research and any changes at all that change the meaning, according to any reader, in any way, would be seen as original research (as DCLawyer apparently interprets the rules).

If we were to take DCLawyer's position literally, I would encourage anyone who has "properly sourced" statements which, according to DCLawyer, are any reference to any article whether or not the article itself justifies the claim or makes it as an offhand comment. Also, the expertise of the author, the subject matter relevance of the journal, the reputation of the journal and even whether or not it is peer-reviewed journal are all, as DCLawyer indicates, irrelevant and to remove such a claim would be removing a "properly sourced" claim. In short, we shouldn't worry about the validity of the source at all because to even question the validity of the source is original research (according to DCLawyer). Clearly, this is not actually how the term "original research" or "properly sourced" should be used. But if this argument continues on a fundamental point about the meaning of the Wikipedia rules, I suggest we seek arbitration. ERosa (talk) 17:04, 16 February 2008 (UTC)[reply]

Please stop the personal attacks on people who are trying to help you. Discuss the article, not the editors, including yourself.

My discussion of recent changes to the article is this. MathDame made a very good first paragraph of the Criticisms section. It cited a recent source that put the criticisms in context. That was followed by material from another recent source that listed the categories of criticisms. That was followed by several paragraphs of mostly very old material about criticisms in those categories. The first paragraph is not the place for that very specific, very old material. The MathDame version is entirely appropriate. Ohio Mailman (talk) 22:21, 16 February 2008 (UTC)[reply]

Its not about personal attacks, its about the rules. You have now done exactly what DCLawyer accused me of - deleting a "properly sourced" reference. I undid your unwarranted removal of the properly sourced material and your attempt to present the reference as an uncontested fact. Your citation was an offhand comment without any supporting evidence from an online journal for software developers. The criticisms were from established and respected journals in the decision sciences (including the journal where Saaty originally published AHP). Your citation certainly did not put anything in context and to claim so is, according to the same rules DCLawyer uses, original research. The age of the mathematical flaws is irrelevant (a mathematical proof doesn't age). IF you have a more recent citation that conclusively disproves the original criticism, then you should state it. But you cannot make any judgement about which is more valid without breaking the rule against OR. Furthermore, the more recent criticism I just added shows how the debate still goes on. Please do not insert POV or OR. Please stick to the Wikipedia rules.ERosa (talk) 23:16, 16 February 2008 (UTC)[reply]

I believe the first paragraph should be the MathDame version: A statement of the state of the world from MSDN, followed by a listing of the areas of criticism from de Steiguer. I believe it should NOT include specific criticisms. They should be aired in the body of the section. Basically, I agree with Ohio Mailman. DCLawyer (talk) 00:56, 17 February 2008 (UTC)[reply]

Thank You. I find the reasons to do otherwise are not very persuasive. Ohio Mailman (talk) 01:00, 17 February 2008 (UTC)[reply]

Be careful that your judgement to do otherwise - or anything - is not original research by the standards DCLawyer applied to others (but apparently not to you). This same situation came up a couple months ago in an exchange with me and DCLawyer and it now appears this uneven application of the rules is used with ERosa. If specific criticisms should not be aired in the first paragraph, neither should specific pro-AHP statements (especially sourced from an online journal that has nothing to do with the decision sciences). It appears that the claim that there is a "general consensus" is what caused this most recent problem. This claim, as it was originally inserted, was lifted directly out of the source and presented as if it were a fact instead of the opinion (the author provided no data to support this claim) from an author of an article in a journal that covers an unrelated topic. My previous attempt to remove other comments were criticized by DCLawyer as removal of a "properly sourced" claim, which, according to him, can never be removed without commiting an infraction against NOR. Pick which rules you want to stick to. If any removal of "properly sourced" claims is a violation of NOR, then you can't remove any specific quote in the first paragraph. If, as you insist that such a claim must be included without any further qualification, then it should not be the first item in the criticisms section. The rest of the article is full of very flowery language about AHP. If you prefer, there is even a subsection at the bottom of this section which is a response to criticisms by proponents where you could insert this claim from the software developers journal. It should not be the first sentence in the criticisms section. I find the reasons to do otherwise as completely illogical and hypocritical.Hubbardaie (talk) 01:18, 17 February 2008 (UTC)[reply]

Actually, now that I've reviewed the history, I'm not sure this "properly sourced" exchange is something I had with DCLawyer but I apparently did have a similar exchange with GoodCop. So I apologize to DCLawyer for accusing him of this. (I'll presume they aren't just socks).Hubbardaie (talk) 01:44, 17 February 2008 (UTC)[reply]

I had a longer response but I lost the data in an update conflict. Actually, yes, the claim about the alleged existance of a supposed general consensus is what much of this debate traces back to. I made one other recent change regarding a new criticism but I haven't seen a debate about that, specifically. I would be interested in seeing what any of you propose as a change. To be clear, I'm not complaining about the inconsiste application of rules (yet) since I haven't seen DCLawyer do anything completely hypocritical. But it would be inconsistent to remove one quote and leave another especially given the relative standing of the sources as journals of decision sciences (Saaty didn't publish his first AHP paper in MSDN) and especially if the removed quote is a criticism in this section on criticisms.ERosa (talk) 01:36, 17 February 2008 (UTC)[reply]

Please stop the personal attacks. Please discuss the article, not the editors or your debates with them.

The article should begin with a statement backed by the MSDN reference. Then it should list the major areas of disagreement, backed by the de Steigeur reference. Then the doors are open to discussing the specific criticisms, as long as they are not given undue weight.

To start the section with the criticisms gives them undue weight, in my opinion. Thousands of serious people use AHP on a regular basis and are satisfied with it. Its use continues to grow. A handful of researchers publish occasional criticisms with strong and possibly valid claims that AHP has serious flaws. The use of AHP continues to grow. By all means describe the criticisms, but give them the weight they deserve as minority views. DCLawyer (talk) 03:20, 17 February 2008 (UTC)[reply]

First, I don't know which personal attacks you are talking about. I see none in my prior statement. Please do not make unfounded accusations. Second, the article should NOT start with the statement "backed" by the online journal which has nothing to do with the decision sciences. To start a section titled "Criticisms" with an actual criticism is not undue weight. I repeat, this is the *criticisms* section. The article itself starts with no criticisms at all. Criticisms are already far down the list and you still want to start even that section with more pro-AHP comments. Your judgement about what gives what undue weight would constitute OR by the same standards you attempt to apply to others. Please apply rules consistently.

And, again, the number of people who use it is irrelevant if the criticisms are formal, mathematical proofs or the result of controlled empirical experiments. As I told you before, that is the common fallacy of Argumentum ad populum. If showing that a lot of people used or believed in something was sufficient proof of anything, imagine what else we could "prove". The fact is that the criticisms listed (with one exception) are from the same journals where Saaty first published AHP. The researchers of mathematical proofs and empirical evidence of errors in AHP are fully aware that lots of people use the method and one listed in his aricle even responds to that. Science is not a vote. Do not resort to your own OR about how many believers a method has to have to be true.

What you are doing is simply making the same response that already exists at the end of the section where responses by proponents are made. Put your MSDN comment there, not at the top of the section. The rest of the entire article is pro-AHP, the published researchers who found specific flaws are given a small section near the end and even then there is a place for the proponents to respond to the critics at the end of the section. To insist that without putting more pro-AHP comments at the top of the one section that mentions any criticisms would constitute undue weight is absurd. Please stop making argumentum ad populum fallacies, claims of personal attacks, and violations of POV and OR.ERosa (talk) 03:53, 17 February 2008 (UTC)[reply]

One more thing. Please, cite an actual survey of researchers in the decision sciences (not merely customers or vendors of AHP, of course) that proves your presumption that critics are a "minority" view among those who actually study the topic professionally. I'm not surprised there are few critics among the customers of AHP. Then they wouldn't be customers. As one of the critics already stated, the proponents are simply unaware of these serious flaws. Please, do not continue to violate the NOR rule by presuming who is the minority. Please cite sources.ERosa (talk) 04:14, 17 February 2008 (UTC)[reply]

A reliable source contains a verifiable statement about consensus and criticisms. That is enough. The statement also rings true^{[citation needed]}, given the voluminous literature about AHP and the numerous users, teachers, and students. There do not seem to be reliable sources which claim that the critics represent any sort of consensus^{[citation needed]}. A few individuals have raised criticisms of AHP. These claims have been published in reliable sources. The criticisms should be heard, but cannot given undue weight. The critics are few in number^{[citation needed]}. Their views, as true (or false) as they might be, are minority views ^{[citation needed]} that have failed to penetrate the larger community of users, teachers, and students. Minority views must not be given undue weight. The first paragraph of the Criticisms section should be the MathDame version. DCLawyer (talk) 04:59, 17 February 2008 (UTC)[reply]

I have not claimed that one side or the other holds the consensus, but you claimed one side is the majority. You have the burden of proof. Again, the number of students and teachers is irrelevant (something about the fallacy of argumentum ad populum is not getting through to you). Again, it is not "undue weight" to actually have criticisms in the criticisms section or even to *start* the criticisms section with criticisms. It would, in fact, give undue weight to the proponents if you start the criticisms section with this quote from the software developers journal (which, when you read it, offers no evidence of the claim). You apparently assume that those published criticisms are the only people who have any criticism and not just representative of some larger group. Unless you think they rate of publication is 100% among all critics (which would make them an unusually prolific group), then there are critics who have not published. Since I found yet another criticism today, I suspect there are even more to be found. In other words, the criticisms are also growing. Let me repeat, if you have a documented survey of academic researchers in this field that show that critics of AHP are a minority, then cite it. Whether you think this quote fom an online journal about software development "rings true" is also OR. Please do not keep promoting your own OR. If you think that any response to any of the mathematical proofs of flaws in AHP conclusively disprove the original criticism, then cite it. Otherwise the proof of the flaw still stands, regardless of the number of users. The first paragraph of the criticisms section should start with criticisms. Pro-AHP comments have plenty of other places to sit in this article. If your point is that all that one needs to start a section is that the claim be "verifiable" (that is, there is a place I can look online where some guy said it, no matter the relevance of the journal to the decision sciences), then any of the criticisms also meet(and exceed) that standard. But I suggest a possible resolution. If you can prove that the MSDN source is actually supposed to be more credible in this field than Management Science or the OR Journal or can prove that critics are actually the minority among decision science researchers, then we can leave the the first sentence as you suggest. To do otherwise is violation of NOR. Please do not violate NOR. ERosa (talk) 05:23, 17 February 2008 (UTC)[reply]

MSDN claims the consensus. That is sufficient. "([S]omething about the fallacy of argumentum ad populum is not getting through to you)" is an uncivil personal attack. Please stop that. DCLawyer (talk) 09:10, 17 February 2008 (UTC)[reply]

It may be sufficient (although I would disagree, given how weak the source is) for inclusion elsewhere in the article, not as the first sentence in the criticisms section. Your actions reveal strong POV. Please refrain from makng further unfounded changes to this material. Your refusal to consider the argumentum ad populum fallacy of your argument is an objective observation. We were having a debate about a topic and I, rightly, pointed out the flaw in your argument. That is a legitimate part of this process. If you choose to take it as a personal insult that is of no concern to me other than it causes you to violate the rule of assumption of good faith. Please cease doing that. ERosa (talk) 13:52, 17 February 2008 (UTC)[reply]

By the way, the sentence you want to include is also within the first sentence of the paragraph I just reverted to. What we are talking about is not even whether the MSDN quote shoud lead the criticisms section. What we are talking about is how you want to make it appear. The next quote "Thus the questions about the validity of AHP are far from having been settled. Also, it is clear that such discussion over a long period has not sufficiently penetrated the AHP users' community, with the result that paper are still being published using a faulty method." is also properly sourced but it is actually from a peer-reviewed decision sciences journal. The contrast between the MSDN article claim of a consensus and the decision science journal quote that the validity of AHP is far from settled is important. To ignore this gives undue weight to a unsubstantiated quote from a software development journal over a properly researched quote from a decision sciences journal. Given that the sentence is already in the top of the first paragraph, given that the source is from a journal that is not in the decision sciences (I can't even tell if it is peer reviewed, given that the quote from the reputable decision science journal contradicts it, given that the rest of the article has plenty of room for this claim, and given that this is the criticisms section, your insistance on a form that only this one particular form of presenting the MSDN quote is acceptable is completely unreasonable. Please do not make Wikipedia your own playground for your own POV.ERosa (talk) 14:17, 17 February 2008 (UTC)[reply]

Be further advised that the countdown for 3RR has already begun. Please do not violate it.ERosa (talk) 14:04, 17 February 2008 (UTC)[reply]

It's actually pretty clear that this process is a mainstream technique in the decision sciences. They teach it at Wharton and Cambridge. There are learned societies devoted to it in Poland, Japan, China, and Indonesia, plus who knows where else. It's also pretty clear that the critics are a vocal minority^{[citation needed]} that doesn't get a lot of ink any more, let alone swing a lot of weight with decision makers. It doesn't make a lot of sense to claim that argumentum ad populum is applicable when the argument is that something is used by a lot of people. 216.183.185.87 (talk) 21:21, 17 February 2008 (UTC)[reply]

Actually, this is exactly the argmetum ad populum fallacy. It doesn't matter how many use it or teach it. The errors found were mathematical proofs or measured in controlle experiments. Such errors do not require a "popular vote" to confirm. These criticisms were published the major decision science journals. Your claim they are even a minority among actual professional reserchers in the decisions sciences has no basis in fact. I'm sure the users of AHP outnumber critics but the vast majority of users are not schooled in the decision sciences and certaintly not published on the topic. The critics listed each have many published works in some of the most respected journals in decision science and they must be given weight in the only section in the article that even mentions criticisms. Again, present any evidence you have that the majority view *among professional researchers in the decision sciences* finds no error with AHP. Once agan, nobody is even saying the quote from the sofware development journal cannot be used. But this article in an online software developers' journal cannot be given weight over several prolific researchers in this field.ERosa (talk) 21:34, 17 February 2008 (UTC)[reply]

Three words: argumentum ad verecundiam Cleome (talk) 23:33, 17 February 2008 (UTC)[reply]

But don't forget that the NOR rule and verifiaility rule mean that the ultimate basis for every article in Wikipedia is appeal to authority. The issue is whether any real authority in this area believes that the number of users overrides a mathematical proof or experimental evidence.ERosa (talk) —Preceding comment was added at 02:30, 18 February 2008 (UTC)[reply]

I think this article might now be past the 3RR and this debate requires some context at this point. Those who till presume a "consensus" in favor of AHP appear to be misunderstanding the nature of this recent flury of edits. From what I've seen so far, the different versions offered both have this quote from MSDN. The difference appears to be how the first quote is framed. In the AHP-has-flaws camp (which includes me and most academics I know), the version presents this quote as a view of proponents - which is accurate - and then contrasts it with a quote from a very respected researcher in a very respected journal. In the AHP-is-so-widely-used-it-must-be-perfect camp, the attempt is to make the issues of flaws in AHP as "settled" in favor of AHP. But its useful to note that I haven't seen anyone here argue AHP is absolutely useless (although a couple of the critics find pretty serious flaws). Its really only an issue of whether AHP has flaws or not. Some of these flaws, like the ones from Dyer, Schenkerman and Perez are a matter of formal, mathematical proofs. The one from Poyhonen is shown experimentally. The flaws are real but may not be debilitating. Note that Saaty himself denied that rank reversal was a problem for years before he decided it needed fixing. If you want a REAL consensus, I think the consensus is that AHP has flaws but still might be useful (although the degree of usefulness does vary according to who you ask. To deny that is has any flaws flies in the face of the mathematical proofs uncovered by these experts.

By the way, I address some problems of AHP in my book "How to Measure Anything" but I recuse myself from adding references to my own book due to COI. It looks like my criticisms might overlap some listed but other criticisms I mention are not yet addressed. One is that AHP seems to make people think they can analyze important decisions without empirical real-world measurements. Another problem is that in practice AHP users are asked to compare factors they can't possibly fully understand. I wrote about one example where a group in an AHP workshop was asked whether they prefer "strategic alignment" or "reduced risk". The users don't really know how much of either of these they are comparing (i.e. how much risk reduction vs. how much alignment) nor even what the terms mean. Unlike quantitative economic/statistical/actuarial modeling methods, AHP users don't appear to encouraged to clarify those points before they choose preferences. If I have any readers reading this, perhaps someone could consider making such a reference. Thanks in advance.Hubbardaie (talk) 03:04, 18 February 2008 (UTC)[reply]

Hubbardaie: 1) This is not your blog. 2) Are you and Erosa the same person? 3) Do you recognize THIS material? DCLawyer (talk) 03:39, 18 February 2008 (UTC)[reply]

First, I didn't intend it to be my blog. Its a talk page about a particular article and I was, like a good Wikipedian, acknowledging my COI but explaining why the reference my be relevant. Second, I don't know ERosa but, since you brought it up, are you the same person as GoodCop? The discussions have some curious similarities plus your page says you are a media lawyer in DC and Goodcop says he does parttime security for media companies. Not conclusive, I admit, but curious. Anyway, I'll assume good faith as the guidelines suggest. Third, I'm aware of the reference and it is straight out of my book. But although Tang clearly has his POV, I only made those comments after citing some of the more respected people in decision science. I'm not sure which parts Tang takes exception to. AHP is, in fact, a subjective weighted scoring method. Rank reversal was even acknowledged by pro-AHP people that it was something that needed to be corrected (thats how Ideal Process Mode got developed). ANd the example of trading "strategic aligment" vs. "risk reduction" was something I personally witnessed in a workshop run by a major AHP tool provider. Anyway, I'll defer more detailed responses to Tang (including responses from his peers who agree with me) for my blog.Hubbardaie (talk) 04:15, 18 February 2008 (UTC)[reply]

Where is your blog? DCLawyer (talk) 05:47, 18 February 2008 (UTC)[reply]

I never started one but apparently I need to. I only felt obliged to respond since you brought it up as part of this discussion about AHP. But thanks for adding my book page to Wikipedia. I might have done that myself but THAT would have been COI. If you meant that me merely arguing in a way consistent to how I was previously publised, I don't believe that is COI itself. If that were the case, then only unpublished people could contribute to WP, or people who contribute to areas other than which they were published. Sounds like lowering the bar a bit. Frankly, it seems like we are really talking about something pretty minor since both versions still put the MSDN quote in the first line. I've made my point.Hubbardaie (talk) 06:01, 18 February 2008 (UTC)[reply]

I Lost data from two edit conflicts so now I've learned to save my work before I submit. I don't think either side has said outright that AHP is useless. This is just about whether it has flaws. In that regard, I don't think many would disagree that it has flaws. At the same time, it is actually used in a lot of places. My personal opinion is that the main flaw is that it is used where sometimes more specialized mathematical methods are already proven to work. And it seems to encourage fact-finding through group meetings as opposed to real-world measurements. But that may just be due to how I see it applied. Our library DOES appear to have Hubbardaie's book so I'll check it out.ChicagoEcon (talk) 04:02, 18 February 2008 (UTC)[reply]