Use of Analytic Hierarchy Process (AHP) methodology to define an academic software to be used by a higher education institution in West Santa Catarina state
Jacir Favrettoa, Luiz Alberto Nottarb
aContestado University, bSanta Catarina State University
ABSTRACT: This article aims to choose an academic management software for a university located in the west portion of the Brazilian state of Santa Catarina. After market research, three companies were pre-selected, which were potentially capable to attend the demands of the institution. They were evaluated according to nine criteria, defined by specialists. With this purpose, a methodology with multiples criteria was adopted (Analytic Hierarchy Process, or AHP), because its flexibility when related to problems in decision making. The analysis demonstrated that the criteria Reliability of Information, Experiences with HEI (Higher Education Institutions), Capacity of Maintainability and Support, and Operational Ease contributed with more than 76% to achieve the goal, and pointed out the company “C” as the best prepared to serve the interests of the Institution.
Key-words: Analytic Hierarchy Process. Academic Management Software. Multiple Criteria Analysis.
1 INTRODUCTION
The human being is challenged all the time to make decisions. The act of deciding most of the times is done automatically, and the choices are taken based on multiple alternatives that are present in everyday life. Then, every choice demands a decision. the etymology of the word decide comes from the Latin decidire, which suggest to abandon one alternative in favor of another. The decision making is an important moment because it aims to set the best alternative or best course of actions to solve an issue (Pereira, 2007).
Different situations in the routine of public or private agents force them to make a decision, even though many times this action is performed intuitively, based only on individual experiences or another subjective parameter. However, many decisions involve complex situations as they contemplate multiple alternatives for the criteria selected. The presence of complex decision problems is considerably common in many areas of knowledge, and the decision makers can solve them using deductive reasoning to validate their choices. One of the methods used for this purpose is the Analytic Hierarchy Process (AHP), which is widely used, simple to operate, and generates reliable results. This method is characterized by the use of quantitative and/or qualitative measurable data, which can be tangible or not, in the analysis of the establish criteria (Saaty, 1990; Vargas, 2010).
AHP is an effective method to be used in decision making and that enables the decision maker to identify the best option among the many possible alternatives, helping this individual to determine the priorities. It also permits the decrease number of complex decision making procedures, transforming them into paired comparative decision processes, based on the structure of the problem, judgments, and synthesis of results (Besteiro et al., 2009).
The higher education institutions are inserted in this scenario and suffer the consequences of the global dynamics. They have necessities, such as the measurement and propagation of results, and the updated maintenance of information as permanent demands from the academic community. To improve their relationship with its community, to keep growing, and to follow up with the demands of local market, universities need to keep an electronic information and communications system that is state-of-art, flexible, and up-to-date. These issues have moved the administration of universities to develop internally or to search in the market the most appropriated technologies to respond quickly and reliably the needs of their users. In these cases, a management software is a tool that can receive, process and store data that can be very useful in the process of development and publication of knowledge (Prieto et al., 2005).
Studies related to multiple criteria problems involving more than one alternative have demonstrated the importance of methodology of hierarchical analysis (AHP) as a solution to complex problems. Costa et al. (2009) applied AHP to a competitive analysis of different statistical software, with the purpose to choose the most adequate product for the studied situation. Galli et al. (2007) used the classical AHP to choose the logistical operator of a telecommunications company, and Gallon et al. (2008) map the managerial tools to evaluate the development of researches in the area of engineering. Weist (2009) used AHP as a support in a comparative analysis of the necessities of business and built guiding factors for a project of services in the area of information technology. The methodology was efficient to determine the best alternative according to the strategies of the business involved.
The aim of this research is to choose the most adequate software for a university, based on AHP. The relevance of this study is to support the best solution to an issue of the institution, contributing to build academic knowledge regarding the use of AHP to solve complex problems, through a practical application of the methodology. The research is characterized as a case study, supported by interviews with specialists linked to the institution, who have strategic positions in the institution. These specialists also contributed to the study by assisting to define the criteria, and the weight of each one. The specialists were also aware of each software company (supplier).
This study is organized in five sections: the first brings a short introduction, the aims and the justification; section 2 presents the theoretical support, followed by the methodological proceedings in section 3; in section 4 there is the case study; in the end, in section 5, there are the final considerations.
2 THEORETICAL REFERRENCES
2.1 The AHP Method
To make a decision regarding different management situations, such as to identify the most adequate use of resources, launch of new products, or the best project are examples of multiple criteria complex problem. Situations such as mentioned before demand a critical analysis of the alternatives and the criteria, permitting to identify the level of strategic alignment of projects or products of the business, which facilitates the assertiveness of the choice (Padovani et al., 2010).
The AHP methodology was created by Thomas Saaty aiming to assist the decision making process and it has the following virtues: a) it is applied to problems with multiple attributes or criteria that are hierarchically structured; b) the methodology analysis the quantitative and the qualitative attributes, incorporating the experience and the preferences of the decision makers; c) it organizes the importance of the attributes and the alternatives; d) AHP can be used in complex situations that demand subjective judgments. Furthermore, it is also adequate to absorb and to deal with the inconsistent analysis of the specialists, suggesting a better evaluation of the problem (Saaty, 1980; 1991).
Marins et al. (2009) demonstrate the importance of supporting methodologies to decision making, highlighting the great versatility and flexibility of AHP. Despite the critics placed by academia in regards to its usage, it is considered that the AHP methodology represents a competitive differential in comparison to its competitors, by stimulating the interaction among all the people involved in the strategy in focus, in many areas, making the studied model much more solid and complete.
AHP has the central premise the structuring of a decision making system that is hierarchically complex in many levels, defined by affinity. The organization of the problem enables a panoramic view of the system, identifying many elements when the problem involves a selection of alternatives permeated by multiple criteria (Cruz Junior et Carvalho, 2003; Iañez et Cunha, 2006).
The main characteristic of the methods with multiple criteria is in searching for the best solution to various possible alternatives, prioritizing the use of resources. The alternatives are ranked according to their priority, based on a set of pre-defined criteria (quantitative or qualitative) organized according to a matrix of decision (Saaty, 1991). Saaty (1980) and Costa (2006) describe the decision making process supported on the following principles:
(i) the construction of a hierarchy – starting from a known problem, it must be structured in hierarchical levels, in order to facilitate the understanding and its evaluation, giving visibility to human reasoning. The key-elements are identified to assist the decision making process and organized according to affinity;
(ii) the definition of priorities – the decision maker, according to his ability, relates the objects to the identified situations, making paired comparisons according to the criteria analyzed;
(iii) the evaluation of consistency – due to the fact there are subjective values, there is a possibility to have inconsistencies from the data acquired from specialists, which must be carefully evaluated through the level of consistency (no less than 0.1).
Costa et al. (2009) suggest to structure the analyzed problem according to the following: at the highest level of hierarchy establish an analytical problem of decision making to be worked with AHP; at the lowest level there are the alternatives to be considered; at the intermediate levels the criteria are established, which can be divided into sub-criteria, or not.
Image 1 presents a model of hierarchy of a problem.
Image 1 – Structure of hierarchical decision process in three levels
Source: Adapted from Costa et al. (2009).
Saaty (1980) remembers that the human being easily relates things and objects and to find similarities based on criteria, pointing the differences and analyzing the intensity of their choices. AHP relates the hierarchical levels comparing the alternatives to the criteria in a paired relationship, revealing the impact of the variables among themselves. The impact of the variables is acquired by the comparison of the variables based on the intensity one over the other, as defined by Saaty (1991) and presented on Table 1.
Table 1 – Numeric classification associated to paired comparisons
Pontuação |
Intensidade |
Forma de avaliação |
1 |
Igual importância |
Os dois parâmetros contribuem igualmente para o objetivo |
3 |
Importância moderada |
Um dos parâmetros é moderadamente favorável sobre o outro |
5 |
Importância forte |
Um dos parâmetros é fortemente favorável sobre o outro |
7 |
Importância muito forte |
Um dos parâmetros é muito mais favorável sobre o outro |
9 |
Importância extrema |
Um dos parâmetros é extremamente favorável sobre o outro |
2,4,6,8 |
Valores intermediários |
Necessidade de definir valores intermediários para os critérios |
AHP is different from other comparative techniques due to the possibility to transform the comparisons, which most of the times are empirical, in numeric values to be processed and compared. The weight of each factor allows to evaluate individually the elements inside the defined hierarchy. This capacity to convert empirical data into a mathematical model is the most important differential in AHP in comparison to other comparative techniques (Gomede et Barros, 2012; Vargas, 2010).
One element is equally important when compared to itself, which means, where the line 1 finds the column 1, the posicion (1,1), results in value 1, as demonstrated in the Table 2. Hence, the main diagonal of the AHP matrix will be always equal to one. The values attributed by specialists are set in the model peer-to-peer, when the evaluator judges using paired references. The judgment of three criteria (C1, C2) is done according to the pairs C1C1; C1C2; C2C1;C2C2.
Table 2 – Comparative matrix (supposing that criterion 1 dominates criterion 2)
Alternativas |
C1 |
C2 |
C1 |
1 |
Avaliação numérica |
C2 |
1/ Avaliação numérica |
1 |
Table 2 presents a model of comparison of analyzed variables (which can contemplate alternatives, criteria and sub-criteria), which diagonal value will always be 1, based on the fact that the intensity of judgment reaches a confrontation peer-to-peer of the same variable. Thus, when facing C1x C1, the result will be 1. If in the comparison between C1 and C2 the evaluator understands the intensity of C1 is 9 times higher than C2, its reciprocal position, C2/C1, will be 1/9, and so on.
As the methodology permits the use of qualitative and quantitative values, there could be some inconsistencies. This occurs due to the fact that values are gathered through the information provided by the specialists, which are subjective and can present inconsistencies in this stage. The verification of consistency of the Matrix of Priorities of Criteria is done by the multiplication of the vector weight, moving to the Matrix of Consistency, with elements w1, w2, w3 and wn. The consistency ratio (CR) can be found through the equation 1, dividing the consistency index (CI) by the random index (RI), a tabulated value set according to the number of criteria, as seen in Table 1. The equations 1 and 2 illustrate how to calculate the CI and the CR (Saaty, 1998). There are specific software that facilitate the calculation of the AHP matrix and that provide the levels of consistency of the evaluations.
(1)
(2)
Table 1 – Values of RI for the Square Matrix in n order
N |
2 |
3 |
4 |
5 |
6 |
7 |
8 |
9 |
10 |
RI |
0 |
0,58 |
0,90 |
1,12 |
1,24 |
1,32 |
1,41 |
1,45 |
1,51 |
The level of consistency is satisfactory, according to Saaty (1991), when CR values are lower than 0.1. When CR values are above 0.1, the author recommends to better re-evaluate the data collected from specialists, observing if they did not miss the results during the process of data collection.
3 METHODOLOGICAL PROCEEDINGS
This article aims to select a software that will improve the communication system of an academic community, and improve the management of information from a university under study. Some alternatives are pre-selected from different providers, and the selection is based on certain criteria for analysis. The development of this research is done through a deductive method, using already established theories to elucidate the goals set. This study is explanatory and with an applicable nature, testing the method with the support of a case study (Cervo et Bervian 2002; Lakatos et Marconi, 2003; Silva et Menezes, 2005). The selection of the software for the university is based on the Analytic Hierarchy Process (AHP) methodology, on criteria previously established by the specialists of the institution, moderated by a facilitator, according to the following steps:
3.1 Definition of the object and the participating specialists in the research
The hierarchical analysis has a starting point with the definition of the object of study by the decision maker. Next, the specialists that will participate in the research must be defined. Such specialists can be found among the professionals and the users that are aware of data management and the information of interest of this study. As an example of specialist, there could be included directors of a company, IT specialists, regional managers, and other information system users.
3.2 Definition of the criteria and the alternatives of research
To establish the most important criteria that are present in this study, and the potential alternative that respond to the necessities of the institution, the knowledge of selected specialists can be used. In this research, the definition of both the objectives and the criteria, as well as the alternatives used in the process of selection can be done by the directors of the institution. It was also included the opinion of teachers and students, who are also involved in the topic. Other institutions that used similar study can be also considered during this stage.
The definition of alternatives can be done from a research with suppliers of the product in the market, or done internally when there is a technical capacity available. Once the objective is defined, the criteria and the alternatives to select the software, a questionnaire is applied in order to define the weight of the criteria and the alternatives.
3.3 Construction of hierarchy, data collection and definition of priorities
The structuring of hierarchy aims to provide a global view of the problem being investigated. The definition of priorities is done through a matrix of binary comparisons, defining the impact of each element related to the referred criterion to the directly superior level. The preferences must be appreciated according to full numbers. The matrix permits to transcribe the value of the factors evaluated and the respective reciprocal (inverse) value placed on the symmetrical position. For such, the elements placed on the left are compared to the elements on the top of the matrix, aiming to achieve a relative impact one over the other (Saaty, 1991).
Ensslin (1998) suggests that the data is collected though brainstorming, by the discussion chaired by the facilitator together with the specialists. Guided by a group of structured questions, the criteria and the alternatives are presented in pairs for the specialists. Based on a scale from 1 to 9 (Table 1), the specialists set values to the criteria that are compared among each other. The same proceeding is done comparing alternatives with each criterion according to the level of importance of each as understood by these specialists.
With the objective to facilitate the understanding and the evaluation of the criteria and the alternatives by the specialists, the problem must be structured in hierarchical levels, as shown in image 1, or as placed in Table 2. Based on the alternatives and on the criteria, the priorities are defined by the specialists, mediated by a facilitator.
Table 2 – Representation of the problem
Foco principal |
Critérios |
Alternativas |
Definição do objetivo |
Critério 1 |
Alternativa 1 Alternativa 2 Alternativa 3 |
Critério 2 |
||
Critério 3 |
||
........ |
||
Critério 9 |
3.4 Modeling and evaluation of data
After data collection, they must be modeled. The calculations can be done mathematically, through natural means or appropriated instruments. For situations that use many criteria and/or alternatives, the manual calculation can be tiring, extensive, and complex. There is a considerable amount of software available in the market, such as the Expert Choice 11.1®, and Assistat (free software), which facilitate the modeling of data and provide a quick response in analyzing the results, achieving a very precise result.
Considering that paired analyses are acquired from the experience of the specialists, Saaty (1980) alerts to the possibility to have inconsistencies. The coherence of hierarchies must be evaluated from the multiplication of each index of coherence by the priority of the corresponding criterion, added to the products. The result must be divided by the same type of evaluation, using the random coherence index corresponding to the dimension of each matrix, weighted by the priorities. In order to achieve this result, it is suggested to measure the Inconsistency Index (II) of a matrix with the support of the equation 1, which must provide a result below 0.1 (Costa et al. 2009; Saaty, 1991; Vargas, 2010).
When the modeling of data is done through the use of software, the coherence index is provided automatically and it can be evaluated if it is in accordance to the methodology, or not. In the case the II found is beyond 10%, Saaty (1980) suggest to go back to the specialists to confirm if there was no mistake in the attribution of values of the paired comparison stage. Vargas (2010) reminds that the II permits only to evaluate the consistency and the regularity of the opinions from the decision makers, which does not guarantee that the opinions are more adequate within the organizational context.
The case study is detailed in the topic.
4 RESULTS
This present study was performed in a university in which coverage area is the west portion and the northern plateau of the Brazilian state of Santa Catarina, in which the Dean’s Office is located in the municipality of Caçador. The university has four campi and more than 8,000 students, considering the undergraduate programs and the graduate ones, divided in lato sensu and stricto sensu levels, in a great number of departments and programs, in many areas of human knowledge. The institution has a series of research programs performed by its vast academic community, in partnership with civil society. Due to its own demand, the university has a considerable amount of daily internet accesses, in online researches, electronic records update from professors and students, besides the regular administrative records of the institution. Based on these questions, the university decided to acquire a new software of academic management, and defined the necessary criteria to find the desired product, using the Hierarchy Analysis Process (AHP) as to help to search for the most adequate product to respond to the needs of its academic community and the general public.
The study performed in this article initiated from the views of the university that required to improve its communications system and information management between its academic community and the general public. It was established that the main goal was to find an information management software to better support the necessities of the public in question. The preference of the institution was to find a product already available in the market.
The study was performed between November and December 2010, with four members of the university, all strategic decision makers from the areas of Information Technology, Administration, and Academic Data.
The steps to build the present research are as following: a) a review of literature in development evaluation, especially the AHP methodology, to support the analysis of the desired results; b) to get in contact with the organization to understand deeply the activities present in the university; c) to know the general goal, the criteria, and the existing alternatives to solve the problem; d) to interview the people in charge of the Information Technology, Administrative, and Academic Data areas; e) to set a new meeting with the interviewees to validate the identifiers and to define the priorities; and f) structuring the model according to the results found in the previous stages.
The scale used to attribute the grades related to the criteria evaluated followed the classification set by Saaty (1991). With the intention to define the most adequate software to fulfill the needs of the university, three suppliers were pre-selected, then compared to the 9 criteria defined by the specialists:
i. Best price; ii. Technology – the system uses state-of-art technology, considering a fully web-based system; iii. Flexibility – it complies with the necessities of the institution, which capacities to create and operate depend only on the present understanding of the final user; iv. Standardizing – the system is capable to classify and standardize documents following the directives of the Brazilian Ministry of Education (MEC) and the directives of the university itself; v. Integration and accessibility – capacity to integrate itself with all areas of the university, enabling quick access and easy understanding; vi. Reliability of information – the information provided by the system match the regimental necessities of local, state, and federal authorities; vii. Experience of the supplier with HEI – demonstrated experience through the presentation of the supplier’s customer portfolio; viii. Operational ease – the system demonstrates practicality and ease when used, without a necessity of advanced understandings to be operated; ix. Capacity of maintenance and support - the company has a qualified technical team that guarantees support; holistic technical capacity of support and maintenance of the system always when needed.
For a safer analysis of the studied object, it is mandatory an appropriate data collection. Based on the criteria and alternatives defined by the university’s specialists, it was used a structured questionnaire mediated by the facilitator, in a brainstorming stage. The facilitator set a meeting in which the goal was to achieve a consensus on the judgment of the participants. The data delated to the criteria is found in Annex 1.
With the values set to the criteria and to the alternatives by the specialists at hand, the data was then modeled. To facilitate and accelerate the calculations, the software Expert Choice® 11.1 was used. This instrument provides the calculations and measures the level of consistency of the evaluations. All evaluations were coherent with the methodology, presenting an II value below than 10%. The results can be seen in Images 2 to 10.
Image 2 – Comparative analysis of the three companies regarding the criterion Best price
Source: The authors themselves.
Images 2 and 3 illustrate the result of the opinion of specialists related to the three companies compared to the criteria Best price and Technology. In regards to the criterion Best price, company A presented significant prevalence over the others, with 73% of the preference of the specialists, followed by company B (18.8%) and company C (8.1%). On the criterion Technology, company C demonstrated better performance, with 66.9% of the preference of specialists, followed by companies B and C, with 24.3% and 8.8%, respectively.
Image 3 – Comparative analysis of the three companies regarding the criterion Technology
Source: The authors themselves.
Image 4 – Comparative analysis of the three companies regarding the criterion Flexibility
Source: The authors themselves.
The performance of the criteria Flexibility and Standardization is shown in Images 4 and 5. For the two criteria evaluated, company C prevailed upon the others, with 65.9% and 64.9% of the preference of evaluators, respectively. The following was company B, with results of 18.5% for the criterion Flexibility, and 27.9% in the criterion Standardization. The lowest performance was seen in company A, with 15.6% in Flexibility, and 7.2% in Standardization.
Image 5 – Comparative analysis of the three companies regarding the criterion Standardization
Source: The authors themselves.
mage 6 presents the performance of the criterion Integration and accessibility, in which company C was the most prominent, with 63.7%, followed by company B (25.8%) and then, company A (10.5%) of the preferences. The same data was found in the analysis of the three companies in the criteria Experience with HEI and Operational ease, as demonstrated in Images 7 and 8.
Image 9 demonstrates the performance of the criteria Reliability of information demonstrated by the three companies, in which the understanding of the specialists, they are all equally dependable, with 33.3% of the results.
Image 6 – Comparative analysis of the three companies regarding the criteria Integration and Accessibility
Source: The authors themselves.
Image 7 – Comparative analysis of the three companies regarding the criterion Experience with HEI
Source: The authors themselves.
Image 8 – Comparative analysis of the three companies regarding the criterion Operational Ease
Source: The authors themselves.
Image 9 – Comparative analysis of the three companies regarding the criterion Reliability of information
Source: The authors themselves.
The last criterion was the Capacity of maintenance and support provided by the three companies, as seen in Image 10. Here, once more, the company C was more emphatic, with 64.9% of the preferences under the evaluation of specialists, followed by company B, with 27.9%, and then company A, with only 7.2%. The analysis also generated a comparative result of the criteria according to the specialists, as seen in Image 11. The global performance of the three companies regarding the set of evaluated criteria is seen in Image 12 and the Annex 2.
Image 10 – Comparative analysis of the three companies regarding the criterion Maintenance and Support Capacity
Source: The authors themselves.
Image 11 – Comparative analysis of the development of the criteria presented
Source: The authors themselves.
Image 11 demonstrates that, four out of nine evaluated criteria responded for 76% of the weight to define the goal, according to the specialists. Reliability of information contributed with 23%, followed by Experience with HEI (21.8%), Capacity of maintenance and support (17.1%), and Operational ease (14.4%). Other criteria presented less proportional impact, being the price charged for the software the less important criterion, causing only 2.3% of impact in the decision process, according to the specialists.
Image 12 – Comparative analysis of the three companies regarding the criterion Best price
Source: The authors themselves.
The final analysis is demonstrated by Image 12, with the opinion of the specialists on the result of the comparison of the alternatives upon the criteria evaluated.
This study showed that company C had higher adherence to the goal established, contributing to 45.7% of the target, followed by company B (28.4%), and in the end, company A (25.9%), based on the opinion of the specialists
4 FINAL CONSIDERATIONS
This article presented the steps to apply the AHP methodology in order to select a software for a university, located in the west region of the Brazilian state of Santa Catarina. The use of the Analytic Hierarchy Process (AHP) is a methodology with a wide use to solve problems that involve multiple criteria and alternatives.
This study originated from the necessity of the institution to improve its communication system and information management with the academic community and the open public. The main objective was to select a company with the best software to respond to the necessities of the university. With the mediation of a moderator, the specialists were chosen – people with notorious understanding regarding the issue, and all of them have some bond with the institution. The specialists established the following criteria as the most important to be evaluated: a) Best price; b) Technology; c) Flexibility; d) Standardization; e) Integration and accessibility; f) Reliability of information; g) Experience with HEI; h) Operational ease; i) Capacity of maintenance and support. The administration of the university decided that the software was going to be searched for in the market, and after a small survey, three companies were set as potentially capable to provide a product that would respond to the necessities of the institution.
It was observed that, from the nine criteria used, four were the most important, with a weight of 76% of impact upon the definition of the goal. The Reliability of information contributed to 23%, followed by Experience with HEO (21.8%), Capacity of maintenance and support (17.1%) and Operational ease (14.4%).
The AHP methodology made evident that, from the nine criteria analyzed, the company C surpassed its concurrent companies in six of those items evaluated, with more than 63% of the preference of the analysts. Only in the criteria Best price and Operational ease the company A demonstrated to be the best when compared to the others. It was also seen that in the criterion Reliability of information the three companies achieved the same score, with a 33.3% of the votes.
The general evaluation of this study identified company C with the best performance of all, with a final result of 45.7%, against 28.4% and 25.9% of the final scores of companies B and A, respectively. Therefore, based on the proposed methodology, it is suggested that the university used the software from company C to manage its information and communication for the users of the services provided by the institution.
Hence, it is understood that the use of the AHP methodology was useful to select the academic software that is most appropriate for the university. One of its virtues was the possibility to analyze nine criteria for three different software companies, making it more organized to select the company that would provide a better response to the necessities of information and structure of the university. As a result, the objective set was reached successfully.
As a proposal for future studies, it is suggested the implementation of tools that permit the academic community to evaluate the performance of the chosen system. Considering the importance of the four first criteria used to define the goal, it is recommended to select the most important criteria though factorial analysis, an analysis of the main components, or another methodology for this end. This would assist the study and would avoid evaluators to be overtired with the process.
REFERENCES
Besteiro, A. M.; Paiva, G.; Miucciato V.; Bueno, J.; Salomon, V. A. P. (2009) A Utilização do método AHP para traçar, como ferramenta para o auxilio a decisão de um candidato, a escolha de um curso de engenharia Universidade Estadual Paulista – UNESP. Disponível em http://www.aedb.br/seget/artigos09/226. Acesso em: 20 novembro 2012.
Costa, J. F. S.; Correia, M. G.; Souza, L. T. T. (2009). Auxílio à decisão utilizando o método AHP: Análise Competitiva Dos Softwares Estatísticos. Bento Gonçalves. 30/08 a 03/09/2009. XLII SBPO. Disponível em: http://www.sobrapo.org.br/sbpo2010/xliisbpo_pdf/72204.pdf. Acesso em: 20 novembro 2010.
Ensslin, L.; Morais, M. L. S. De; Petri, S. M. (1998). Construção de um Modelo Multicritério em Apoio ao Processo Decisório na Compra de um Computador. UFSC, Florianópolis, ENEGEP1998, acesso: www.abepro.org.br, em 15/01/2011.
Galli, I.;Nascimento, L. P.Da S.; Belderrain, M. C.N. (2007). Aplicação do método AHP clássico na escolha do operador logístico de uma empresa de telecomunicações. ENEGEP. Foz do Iguacu, PR, 09 de outubro de 2007. Disponível em www.ic.uff.br/~emitacc, acesso em 03 março 2014.
Gallon, A. V.; Nascimento, S. Do; Ensslin, L.; Dutra, A. (2014). Mapeamento das ferramentas gerenciais para avaliação de desempenho disseminadas em pesquisas da área de engenharia. Disponível em www.revista-ped.unifei.edu.br/documentos/V06N01/n07_art04.pdf. Acesso em 02 fevereiro 2014.
Gomede, E; Barros, R. M. (2012). Utilizando o método Analytic Hierarchy Process (AHP) para priorização de serviços de TI: Um estudo de caso. VIII Simpósio Brasileiro de Sistemas de Informação (SBSI) Trilhas Técnicas. Londrina.
Marins, C. S.; Souza, D. O.; Barros, M. S. (2009). O uso do método de análise hierárquica (AHP) na tomada de decisões gerenciais – um estudo de caso. XLI SBPO 2009 – Simpósio Brasileiro de Pesquisa Operacional. 1 a 4 de setembro de 2009. Porto Seguro. Bahia.
Pereira, S. D. L. (2007). Modelagem para Decisão por Pacotes e Viagem: Um Estudo de Caso Usando Processo de Hierarquia Analítica – AHP. Natal- RN, 2007.
Prieto, V. C., Laurindo, F. J. B.; Carvalho, M. M. (2005). Método de análisis jerárquico aplicado a la selección de ambientes de aprendizaje: Estudio de caso en el área de la enseñanza superior a distancia. Caracas, 2005. Espacios. Vol. 26 (2) 2005. Disponível em: http://www.revistaespacios.com/a05v26n02/05260201.html. Acesso em 12 novembro 2010.
Saaty, T.L. (1980). The Analytic Hierarchy Process. New York: McGraw-Hill.
Saaty, T.L. (1991). Método de Análise Hierárquica. São Paulo, Editora Makron.
Vargas, R. V. (2010). Utilizando a programação Multicritério (Analytic Hierarchy Process –AHP) para selecionar e priorizar projetos na gestão de portfólio. PMI Global Congress 2010 – North America. Washington –DC – EUA- 2010. Disponível em www.ricardo-vargas.com. Acesso em 16 janeiro 2014.
Weist, P. (2009). An AHP-based Decision Making Framework for IT Service Design. MWAIS 2009 Proceedings. Paper 11. http://aisel.aisnet.org/mwais2009/11.Acesso em 20 novembro 2010.