Evaluation of Business Intelligence in telecommunications companies
Breno Brand Fernandes
brenobfernandes@gmail.com
TemboSocial Inc, Toronto, Canada
Priscilla Cristina Cabral Ribeiro
priscillaribeiro@id.uff.br
Fluminense Federal University – UFF, Niterói, Rio de Janeiro, Brazil
Helder Gomes Costa
heldergc@id.uff.br
Fluminense Federal University – UFF, Niterói, Rio de Janeiro, Brazil
ABSTRACT
Highlights: Most companies have issues with large volumes of data, lack of information, knowledge, and insufficient reporting. Business Intelligence (BI) allows companies to find patterns and connections between apparently independent and disconnected pieces of data. Evaluating success or effectiveness of information systems is crucial for investment in these technologies. Despite advancements made by evaluation studies, there is still a gap in regard to BI-oriented evaluation models.
Objective: Propose and apply a model for BI evaluation in telecommunication companies through IT evaluation attributes. Design/Methodology/Approach: In order to develop the research, an alignment between literature review and field research was made. Multiple case studies of qualitative approach were used as method, using semi-structured interviews as data collection techniques. These were conducted with IT managers and system users.
Results: Despite the main advantages, BI's disadvantages and aspects for obtaining success were identified in literature and are aligned with the field research. It was found that motivations and pressures for BI implementation are principally related to the alignment of the organization's strategic planning to BI's benefits.
Investigation's limitations: Not all hardware attributes can be applied to software evaluation processes, similar to how some are not appropriate to the service sector. Thus, this work's proposition focuses on software evaluation attributes.
Practical implications: From a management perspective, this article contributes with the proposition of a new set of key attributes to the evaluation of BI after its implementation in companies which are not necessarily in the telecommunications sector.
Originality/Value: Despite the adoption and growth of BI, there is a gap to be filled regarding models for the technology evaluation. This type of model is crucial for understanding why a tool with a number of advantages is still affected by implementation difficulties and use in the telecommunications sector. Therefore, this work provides a set of evaluation attributes supported by literature which can subsidize the development of new evaluative studies that are not restricted to IT.
Keywords: Evaluation; Model; Business Intelligence; Telecommunication.
1 INTRODUCTION
Most companies have issues with large volumes of data, lack of information, knowledge, and insufficient reporting (Farrokhi, 2012; Gandomi et Haider, 2014; Alpar et al., 2015). Meanwhile, executives prefer to work with singular and integrated information instead of a larger number of reports originating from different information systems (Ferreira et Kuniyoshi, 2015). In the case of small and medium businesses, the businessperson or executive needs to know the impact adequate information management can have on the organization’s performance. (Sanchez Limón et De la Garza Cárdenas, 2018). This allows competition in better conditions through enabling cost reduction, quality improvement, shorter deadlines, product diversification, and better post-sales service (Leal Morantes et al., 2018).
In this context, Business Intelligence (BI) appears as a tool of integration, transformation, interpretation, and viewing of this data (Duan et Xu, 2012; Chen et al., 2012). BI allows companies to find patterns and connections between apparently independent and disconnected pieces of data. This allows for there to be new answers to the organization’s needs and the creation of fundamental information to decision-making (Kowalczyk et al., 2013; Chaudhuri et al., 2011).
In order to decrease investment risks in information technology (IT) that relate to lack of alignment between the technology and business strategies, it is necessary to have an effective evaluation policy or a set of guidelines that follow these investments (Lönnqvist et Pirttimäki, 2006; Ribeiro, 2010). For Delone et McLean (2003). The evaluation of success or effectiveness of information systems is fundamental to investing in these technologies.
Considering this, the main research question is: how to evaluate BI in companies of the telecommunications sector? This central question leads to the following secondary questions: what are BI’s advantages and disadvantages? What are the pressures and motivations to the implementation of BI?
Despite the advancements already made with the studies developed, there is still a gap regarding dedicated models, specifically relating to the evaluation of BI (Popovič et al., 2014; Bole et al., 2015). Taking this into consideration, this work’s general objective is to propose and apply a model for BI evaluation in telecommunication companies through IT evaluation attributes. This contribution lies in filling this gap by proposing a model to evaluate BI.
2 METHODOLOGY
The methodological stages of the research were structured and organized, aiming at reaching the answer to the main research question, which in turn was reached through obtaining answers to the secondary questions. These were obtained through developing the actions indicated in Figure 1:
Figure 1. Research secondary questions and actions conducted to answer them.
In order to answer the main research question, articles found in the bibliographic review and exploratory research were used. They were organized into two groups: IT evaluation and BI evaluation. The first group was split into two perspectives: the financial one and the most complex evaluation models. In the first one, contributions to building the model were found, as it will be disclosed ahead. In the second, the contributions came from two studies from DeLone et McLean (1992; 2003)—which display a great number of citations in the leading IT journals (11.654 and 9.654 citations on Google Scholar, respectively, by August 2018). The majority of subattributes were extracted from them (23 out of the 21 cited here were mentioned by them).
In the second group, BI evaluation, another set of authors was utilized. Among the ones consulted, the main ones were selected after their publications had been read, their contribution had been analyzed, and their relevance to the research had been verified Figure 4). To organize the variables, the method proposed by Ribeiro (2010) was utilized with the goal of organizing the contributions to attributes and subattributes and the area of operation (IT or BI). The answers to the main question are indicated in Figures 9 and 10.
Regarding the field research, the semi-structured question script was utilized to answer the questions in Figure 1. Likert’s scale was used as a scale for the answers to these questions, from 1 to 5 associated to it for classification (1 = very low; 2 = low; 3 = average; 4 = high; 5 = very high). In Figure 9 the interviewees’ scores and their sum are indicated in the last column. Therefore, BI’s evaluation by the respondents is known through close-ended questions. In Figure 10 the analysis of BI by its score frequency was preferred in order to conclude whether it obtained a positive score (greater number of “4” and “5” answers), an average score (greater number of “3” answers) or if it needs to improve (greater number of “1” and “2” answers).
After the questionnaire’s data collection, the subattributes Complexity and Integration of the attribute System’s Quality had their scales inverted because differently from all the other subattributes, the greater weight given in the answer, the more negatively the answer will affect BI’s evaluation. In order to process the registered information, the following criterion was adopted for answers of interviewees from the same organization: the interviewees’ answers (scores from 1 to 5) given to a single subattribute were subtracted one from the other. Results superior to |1| were considered divergent answers.
3 DEVELOPMENT AND RESULTS
The literature review and research results will be shown in this item.
3.1 Literature review
The literature review was conducted in order for answers to be obtained for the questions in Figure 1. The process chosen for the literature review has as basis the article by de Carvalho Pereira et al. (2017) (with a few steps being independent from the model utilized by the authors). The bases utilized for the research were articles indexed on the databases Web of Science (ISI) and Scopus, resulting in an 85-article set. The next stage consisted of manual verification of titles and summaries for the verification of adherence of the articles selected to the theme. The starting point for the literature review is a sample with 57 articles that served as basis for the model construction. The reason is that after filtering, a few articles that had been excluded during the filtering process were restored to the sample due to their adherence to the theme. From that collection, some points were raised for this review, such as the advantages shown in Figure 2.
Figure 2. Advantages of BI.
Despite the fact that the adoption of BI and its set of techniques brings several advantages as previously stated, there is risk both in adopting and in not adopting BI. In any relevant investment a few points must be observed, such as: possibility of early return and impact on competitiveness, effects of the adoption on the organization’s internal processes; opportunity cost of other investments, and the possibility that the tool will undermine another investment or render it useless (Isaca, 2014). From the texts analyzed in this research Figure 3 was constructed; it summarizes the disadvantages of BI.
Figure 3. Disadvantages of BI.
Additionally, the literature review sought to identify the characteristics that a BI evaluation model must have. Figure 4 synthetizes the search results.
Figure 4. Attributes and subattributes for BI and IT evaluation.
3.2 Field research
The field research aimed at evaluating BI adoption in Brazilian companies operating in the Brazilian telecommunications sector.
3.2.1 Sample
Semi-structured interviews in companies that operate in the telecommunications sector in Brazil were conducted for data collection and analysis. The data was collected through a research tool consisting of an open-ended and close-ended questions script. Likert’s scale was used as a scale for the answers to these questions; it is symmetric and balanced, allowing one to measure the degree of agreement with the questions asked to the respondent through five “stance areas” (Likert, 1932). The scale ranges from 1 to 5; 1 = very low; 2 = low; 3 = average; 4 = high; 5 = very high. The unities of analysis were three companies whose characteristics are summarized in Figure 5.
Figure 5. Characteristics of the companies investigated.
Caption: NA = not answered
In each researched organization two employees were chosen, aiming at contemplating both perspectives visualized by BI: technique, composed of the set of IT tools that constitute it, represented by the IT manager, the provider; and management, which considers all the benefits BI brings the organization, represented by the organization’s manager, who is the user of the information extracted from the system (Pirttimäki et al., 2006). The interviewee’s characteristics are summarized in Figure 6.
Figure 6. Interviewees’ profiles.
3.2.2 Data analysis—open-ended questions
At the first stage of field research, the interviewees were asked three open-ended questions: (a) In your perception, what were the advantages of adopting BI?; (b) In your perception, what were the disadvantages of adopting BI?; (c) In your perception, what were the main pressures and motivations for implementing BI in your company? Figure 7 shows a compilation of the answers given by the interviewees, comparing the advantages (Figure 2) and disadvantages reported in literature (Figure 3), considering questions a and b.
Figure 7. Advantages and disadvantages perceived by the companies reported in literature.
Obs. Advantages and disadvantages indicated in Figures 2 and 3
The analysis of Figure 7 shows:
- With respect to the advantages of BI, only three of those mentioned in literature were aligned with all the case studies;
- Only company A noticed in its operation all the advantages found in literature, excluding standardization;
- Only the disadvantages that relate to training and report archiving found in literature were observed in the field research.
The last aspect raised as a disadvantage in Figure 3 (difficulty by some BI system users in dealing with large amounts of information, and archiving these reports due to physical limitations) was perceived as motivation by companies to adopt BI. As a solution, company B designed a procedure oriented at reporting according to the business’s demands. One of the problems indicated in literature is the physical limitation on report archiving (Schulz et al., 2015). On this front, Alpar et al. (2015) suggest reutilizing and sharing reports among the organization’s areas, a practice adopted by the three companies studied here.
Figure 8 shows the compilation of answers to question (c): “In your perception, what were the main pressures and motivations for implementing BI in your company?” and the comparison to the results found in the literature review. Due to company A’s size, the pressure to stay afloat did not influence its decision to implement BI.
Figure 8. Motivations to implement BI.
3.2.3 Results analysis—close-ended questions
Here the answers obtained via a close-ended questionnaire are analyzed through the contrast between the answers of the IT manager and the system user’s. It should be noted that letter “M” was used to indicate the manager’s answers whereas “U” indicates the user’s.
In Figure 9 one can notice that the IT managers and users’ answers (score/evaluation) showed little divergence, even though there is different scoring in most attributes. This is due to the plurality of the user profiles, since this groups of users contains economists, advertisers, technical support agents, and IT managers. Previous experiences can lead to different scoring, given that most divergences happened due to lack of technical knowledge by the users. This affected other questions, such as easy learning, intended use, and voluntary use.
Figure 9. Field-collected data: Scoring by the interviewees evaluating BI.
Caption: M = IT manager; U = User
According to Moore et Benbasat (1991), the attribute “voluntary use” is connected to the quality perceived by the user. Thus, since the user considered it difficult to generate reports, as stated in the open-ended questions, they distanced themselves from the system, therefore affecting its voluntary use.
The subattribute “complexity” had the lowest total score at 16 points. This attribute is generally linked to ease of use. Access level initially seems to be a specific characteristic of company B, where it scored low, and is an important attribute for BI success, according to Işik et al. (2013). With extended use of BI—from, the strategic sphere to operations—the amount of reporting experienced noticeable growth. Alpar et al. (2015) suggest report reutilization and sharing among areas of the organization as a solution to this problem. All the companies from the study cases archive and reutilize or share their reports among the organization’s areas as literature suggests. The subattribute “better results” had the highest total score at 29 points. It was followed by up-to-dateness and accuracy, at 28 each. It can be said that, as a general rule, the benefits of BI extracted from the field research are aligned with the literature.
While Figure 9 shows data collected in the field from a horizontal perspective, once the data is transformed into frequency perspective, as seen in Figure 10, vertically, it is credible that 80% of answers had positive scoring, 4 and 5. 50% of answers were concentrated at high score (4) and 30% at very high (5). The analysis of BI through the presented scale leads one to assume BI was evaluated as above average in the three companies studied, in spite of the identified disadvantages.
Figure 10. Data from the study cases from a frequency perspective.
4 CONCLUSION
It was verified that the advantages and benefits of BI reported in literature as key to its evaluation were confirmed by the case study participants. BI is acknowledged as a strategic tool that centralizes information in real time (updated information) from various perspectives, ensuring a better understanding of the organization, which results in faster decisions of higher quality. This information is normally presented through reports with dashboards and indicators that facilitate its interpretation.
With regard to disadvantages, the concept of BI 2.0, a trend according to scientific literature, was scarcely identified in practice. Only one of the organizations showed, in some aspect, the existence of interaction between company, providers, and clients in a single BI system. Even when it was identified, interviewees diverged about its functionality. The other disadvantage found was the difficulty in selecting which report to use in the decision-making, given the large amount of information available through BI. As a solution to this problem, one of the organizations developed a procedure oriented at generating and selecting reports. The complexity of using BI, highlighted in this research, caused other aspects, such as ease use and voluntary use, to be negatively affected, resulting in another negative perspective of BI. This setting requires greater attention to the training of users and managers in order to diminish this negative view so it does not affect the use experience.
With data decentralization inside organizations, reporting with information in real time rise as the main motivation to implement BI. Integrating data in a system is crucial because it not only adds quality to decision-making, but also reduces costs and time otherwise directed at manually generating reports that integrate different data. The large volume of data, lack of information on time and insufficient reporting are elements that act as forces pushing toward the adoption of BI. Another force is the organization’s strategic alignment with investing in technology for the organization’s evolution and survival.
A number of requirements were identified as decisive for the success of BI, such as: previous experience with the analytic decision process; the identification of whether there is alignment between the benefits BI can offer the organization and its strategic planning; support from the board of directors; and a leader figure in the organization that establishes communication between BI’s technical part and the organization’s other departments.
According to the case studies, even without financial metrics, that is, without conducting a financial study about BI’s ROI, its viability through intangible concepts suffices for the decision to implement it. The increase in the organization’s value due to BI is sufficient for its implementation. Therefore, it is critical that one assesses whether the benefits of BI align with the organization’s strategic planning. The implementation in each of the three companies investigated was successful, despite its having been carried out from a project of little structure. This data suggests that its implementation does not necessarily demand a complex project, which can be justified by the low level of complexity in implementing this type of system, that injects greater work in the stage at which data from various sources is unified.
For future studies, it is suggested that a structured survey be conducted with IT experts, preferably managers, who were involved in the process of implementing BI in technology companies.
The research aims at evaluating IT inasmuch as investment in this area in private and public companies can be overrated or underrated, if the product to which the institution’s or society’s resources are allocated is not evaluated.
Considering the literature review, other researchers can have a foundation on which to develop their studies. The set of attributes identified in the research of the leading authors on technology and information systems evaluation models allows readers to understand and reproduce this proposition; alternatively, they can develop another one based on the information available here. Furthermore, the model presented here can contribute to the creation of other evaluation models not exclusively aimed at IT.
REFERENCES
Abelló, A., Romero, O., Pedersen, T. B., Berlanga, R., Nebot, v., Aramburu, M. Simitsis, A. (2015), “Using semantic web technologies for exploratory OLAP: A survey”, IEEE Transactions on Knowledge and Data Engineering, Vol. 27, No. 2, pp. 571–588. http://dx.doi.org/10.1109/TKDE.2014.2330822.
Affeldt, F. S.; Silva, S. D. da. (2013), “Information Architecture Analysis Using Business Intelligence Tools Based on the Information Needs of Executives”, Journal of Information Systems and Technology Management: JISTEM, Vol. 10, No. 2, pp. 251–270.
Agarwal, R.; Prasad, J. (1997), “The role of innovation characteristics and perceived voluntariness in the acceptance of information technologies”, Decision Sciences, Vol. 28, No. 3, pp. 557–582.
Alpar, P.; Engler, T. H.; Schulz, M. (2015), “Influence of social software features on the reuse of Business Intelligence reports. Information”, Processing and Management, Vol. 51, No. 3, pp. 235–251.
Azvine, B.; Cui, Z.; Nauck, D. D. (2005), “Towards real-time business intelligence”, BT Technology Journal, Vol. 23, No. 3, pp. 214–225.
Bailey, J. E.; Pearson, S. W. (1983), “Development of a Tool for Measuring and Analyzing Computer User Satisfaction”, Management Science, Vol. 29, No. 5, pp. 530–545.
Belardo, S.; Karwan, K. R.; Wallace, W. A. (1982), “DSS component design through field experimentation: an application to emergency management”, artigo apresentado no International Conference on Information Systems, Ann Arbor, Michigan, 1982.
Boland, G. W., Thrall, J. H., e Duszak, R. (2015), “Business intelligence, data mining, and future trends”, Journal of the American College of Radiology, Vol. 12, No. 1, pp. 9-11.
Bole, U., Popovič, A., Žabkar, J., Papa, G., e Jaklič, J. (2015), “A case analysis of embryonic data mining success”, International journal of information management, Vol. 35, No. 2, pp. 253-259.
Bravo Orellana, E. Santana Ormeño, M., Rodón Mòdol, J. (2014), “Impacto de la automatización sobre el desempeño: Evaluación en sistemas de información”, Revista Venezolana de Gerencia (RVG), Vol. 19, No. 66, pp. 267-286.
Brooks, P.; El-Gayar, O.; Sarnikar, S. (2015), “A framework for developing a domain specific business intelligence maturity model: Application to healthcare”, International Journal of Information Management, Vol. 35, No. 3, pp. 337–345.
Burton-Jones, A.; McLean, E. R.; Monod, E. (2014), “Theoretical Perspectives in IS Research: From Variance and Process to Conceptual Latitude and Conceptual Fit”, European Journal of Information Systems, Vol. 24, No. 6, pp. 664-679.
Carvalho, T. V. de; Sassi, R. J. (2013), “Business Intelligence as a competitive advantage on a Brazilian chemical industry in the global crisis of 2008, 2009 and 2010”, Advances in Chemical, Material and Metallurgical Engineering, Vol. 634-638, pp. 3883–3886. https://doi.org/10.4028/www.scientific.net/AMR.634-638.3883
Chaudhuri, S.; Dayal, U.; Narasayya, V. (2011), “An overview of business intelligence technology”, Communications of the ACM, Vol. 54, No. 8, pp. 88-98.
Chen, H.; Chiang, R. H. L.; Storey, V. C. (2012), “Business Intelligence and Analytics: From Big Data to Big Impact”, MIS Quarterly, Vol. 36, No. 4, pp. 1165–1188.
Chen, J. V., Jubilado, R. J. M., Capistrano, E. P. S., e Yen, D. C. (2015), “Factors affecting online tax filing–An application of the IS Success Model and trust theory”, Computers in Human Behavior, Vol. 43, pp. 251-262.
Chen, J. V., Rungruengsamrit, D., Rajkumar, T. M., e Yen, D. C. (2013a), “Success of electronic commerce Web sites: A comparative study in two countries”, Information e management, Vol. 50, No. 6, pp. 344-355.
Chen, J. V.; Chen, Y.; Capistrano, E. P. S. (2013b), “Process quality and collaboration quality on B2B e-commerce”, Industrial Management and Data Systems, Vol. 113, No. 6, pp. 908–926.
Chvatalova, Z.; Koch, M. (2015), “Optimizing of Information Systems in Companies: Support of Sustainable Performance”, Procedia - Social and Behavioral Sciences, Vol. 213, pp. 842–847.
Cook, A.; Davis, C. (2003), “Shifting gears to accommodate diversity: how and why an information systems project manager should customize leadership style to suit multicultural teams”, artigo apresentado no SWDSI 2003: Southwest Decision Sciences Institute Conference, 2-5 de mar. 2003.
Davis, F. D. (1989), “Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology”, MIS Quarterly, Vol. 13, No. 3, pp. 319–340.
Delone, W. H.; Mclean, E. R. (1992), “Information Systems Success: The Quest for the Dependent Variable”, Information Systems Research, Vol. 3, No. 1, pp. 60–95.
Delone, W. H.; Mclean, E. R. (2003), “The DeLone and McLean Model of Information Systems Success: A Ten-Year Update”, Journal of Management Information Systems, Vol. 19, No., 4, pp. 9–30.
Dickson, G. W.; Desanctis, G.; Mcbride, D. J. (1986), “Understanding the effectiveness of computer graphics for decision support: A cumulative experimental approach”, Communications of the ACM, Vol. 29, No. 1, pp. 40–47.
Dickson, G. W.; Senn, J. A.; Chervany, N. L. (1977), “Research in Management Information Systems: The Minnesota Experiments”, Management Science, Vol. 23, No. 9, pp. 913–934.
Duan, L.; Xu, L. Da. (2012), “Business intelligence for enterprise systems: A survey”, IEEE Transactions on Industrial Informatics, Vol. 8, No. 3, pp. 679-687.
Farrokhi, V. (2012), “The necessities for building a model to evaluate Business Intelligence projects- Literature Rev.”, International Journal of Computer Science e Engineering Survey, Vol. 3, No. 2, pp. 1–10.
Ferreira, A. A.; Kuniyoshi, M. S. (2015), “Critical factors in the implementation process of integrated management systems”, Journal of Information Systems and Technology Management, Vol. 12, No. 1, pp. 145–164.
Gandomi, A.; Haider, M. (2014), “Beyond the hype: Big data concepts, methods, and analytics”, International Journal of Information Management, Vol. 35, No. 2, pp. 137–144.
Ghazanfari, M.; Jafari, M.; Rouhani, S. (2011), “A tool to evaluate the business intelligence of enterprise systems”, Scientia Iranica, Transaction E, Industrial Engineering, Vol. 18, No. 6, pp. 1579–1590.
Golfarelli, M., Mandreoli, F., Penzo, W., Rizzi, S., e Turricchia, E. (2012), “OLAP query reformulation in peer-to-peer data warehousing”, Information Systems, Vol. 37, No. 5, pp. 393-411.
Hamilton, S.; Chervany, N. L. (1981), “Evaluating Information System Effectiveness - Part I: Comparing Evaluation Approaches”, MIS Quarterly, Vol. 5, No. 3, pp. 55–69.
Hannula, M.; Pirttimäki, V. (2003), “Business intelligence empirical study on the top 50 Finnish companies”, Journal of American Academy of Business, Vol. 2, pp. 593–599.
IBM (2013). What will we make of this moment? 2013 IBM Annual Report. Disponível em: https://www.ibm.com/annualreport/2013/bin/assets/2013_ibm_annual.pdf (Acesso em: 26 ago. 2015.
Isaca. (2013), “Big Data: Impacts and Benefits”, ISACA, pp. 1–14.
Isaca. (2014), “Generating Value from Big Data Analytics”, ISACA, pp. 1–12.
Işik, Ö.; Jones, M. C.; Sidorova, A. (2013), “Business intelligence success: The roles of BI capabilities and decision environments”, Information and Management, Vol. 50, No. 1, pp. 13–23.
Jarvenpaa, S.; Dickson, G.; Desanctis, G. (1985), “Methodological Issues in Experimental IS Research: Experiences and Recommendations”, MIS Quarterly, Vol. 9, No. 2, pp. 141–157.
Kettinger, W. J.; Lee, C. C. (1994), “Perceived Service Quality and User Satisfaction with the Information Services Function”, Decision Sciences, Vol. 25, Nos. 5-6, pp. 737–766.
Kowalczyk, M.; Buxmann, P. (2015), “An ambidextrous perspective on business intelligence and analytics support in decision processes: Insights from a multiple case study”, Decision Support Systems, Vol. 80, pp. 1–13.
Kowalczyk, M.; Buxmann, P.; Besier, J. (2013), “Investigating Business Intelligence and Analytics from a Decision Process Perspective: A Structured Literature Review”, artigo apresentado no ECIS, European Conference on Information Systems, Utrecht, Holanda, 5-8 jun. 2013
Kubina, M.; Koman, G.; Kubinova, I. (2015), “Possibility of Improving Efficiency within Business Intelligence Systems in Companies”, Procedia Economics and Finance, Vol. 26, No. 15, pp. 300–305.
Kumar, A.; Shankar, R.; Debnath, R. M. (2015), “Analyzing customer preference and measuring relative efficiency in telecom sector: A hybrid fuzzy AHP/DEA study”, Telematics and Informatics, Vol. 32, No. 3, pp. 447–462.
Kumar, D. P. (2012), “Impact of Business Intelligence Systems in Indian Telecom Industry”, Business Intelligence Journal, Vol. 5, No. 2, pp. 358-366.
Leal Morantes, M. E., Labarca Ferrer, N. J., Bracho Parra, O. del S., Vargas de Hernández, V. E. (2018), “Gestión Tecnológica en pymes del sector textil del municipio Maracaibo- estado Zulia Venezuela”, Revista Venezolana de Gerencia, Vol. 23, No. 82, pp. 314-335.
Likert, R. (1932), “A technique for the measurement of attitudes”, Archives of psychology, R.S. Woodworth, New York.
Lin, C.; Pervan, G. (2003), “The practice of IS/IT benefits management in large Australian organizations”, Information and Management, Vol. 41, No. 1, pp. 13–24.
Lönnqvist, A.; Pirttimäki, V. (2006), “The Measurement of Business Intelligence”, Information Systems Management, Vol. 23, No. 1, pp. 32–40.
Lucas, H. C. (1975), “Why information systems fail”. Columbia University Press, New York.
Maish, A. M. (1979), “A User’s Behavior Toward His MIS”, MIS Quarterly, Vol. 3, No. 1, pp. 39–52.
Marinheiro, A.; Bernardino, J. (2015), “Experimental Evaluation of Open Source Business Intelligence Suites using OpenBRR.” IEEE Latin America Transactions, Vol., 13, No. 3, pp. 810–817.
Marius, G.; Aref, M.; Bilal, H. (2009), “Real time on-line analytical processing for business intelligence”, UPB Scientific Bulletin, Series C: Electrical Engineering, Vol. 71, No. 3, pp. 79–88.
Marthandan, G.; Tang, C. M. (2010), “Information technology evaluation: issues and challenges”, Journal of Systems and Information Technology, Vol. 12, No. 1, pp. 37–55.
Mohammadi, H. (2015), “Investigating users’ perspectives on e-learning: An integration of TAM and IS success model”, Computers in Human Behavior, Vol. 45, pp. 359–374.
Moore, G. C.; Benbasat, I. (1991), “Development of an instrument to measure the perceptions of adopting an information technology innovation”, Information Systems Research, Vol. 2, No. 3, pp. 192–222.
Munro, M. C.; Davis, G. B. (1977), “Determining Management Information Needs: A Comparison of Methods”, MIS Quarterly, Vol. 1, pp. 55–67.
Nanavati, A. A., Singh, R., Chakraborty, D., Dasgupta, K., Mukherjea, S., Das, G., ... e Joshi, A. (2008), “Analyzing the structure and evolution of massive telecom graphs”, IEEE Transactions on Knowledge and Data Engineering, Vol. 20, No. 5, pp. 703-718.
Olson, M. H.; Lucas, H. C.; Kling, R. (1982), “The Impact of Office Automation on the Organization: Some Implications for Research and Practice”, Communications of the ACM, Vol. 25, No. 11, pp. 838–847.
Pendharkar, P. C. (2010), “Valuing interdependent multi-stage IT investments: A real options approach”, European Journal of Operational Research, Vol. 201, No. 3, pp. 847–859.
Pereira, F. C.; Costa, H. G.; Pereira, V. (2017). “Patent filings versus articles published: A review of the literature in the context of Multicriteria Decision Aid”. World Patent Information, Vol. 50, pp. 17-26.
Petter, S.; Delone, W.; Mclean, E. (2008), “Measuring information systems success: models, dimensions, measures, and interrelationships”, European Journal of Information Systems, Vol. 17, No. 3, pp. 236–263.
Pirttimäki, V.; Lönnqvist, A.; Karjaluoto, A. (2006), “Measurement of Business Intelligence in a Finnish Telecom- munications Company” Electronic Journal Of Knowledge Management, Vol. 4, No. 1, pp. 83–90.
Pitt, L. F.; Watson, R. T.; Kavan, C. B. (1995), “Service Quality: A Measure of Information Systems Effectiveness”. MIS Quarterly, Vol. 19, No. 2, pp. 173–187.
Popovič, A., Hackney, R., Coelho, P. S., e Jaklič, J. (2014), “How information-sharing values influence the use of information systems: An investigation in the business intelligence systems context”, The Journal of Strategic Information Systems, Vol. 23, No. 4, pp. 270-283.
Ramakrishnan, T.; Jones, M. C.; Sidorova, A. (2012), “Factors influencing business intelligence (BI) data collection strategies: An empirical investigation”, Decision Support Systems, Vol. 52, No. 2, pp. 486–496.
Ribeiro, P. C. C. (2010), Proposição de um método de avaliação de tecnologia de identificação: O caso RFID nas cadeias de carne bovina no Brasil e nos EUA. Tese de Doutorado em Engenharia de Produção, Programa de Pós Gradução em Engenharia de Produção, Universidade Federal de São Carlos, São Carlos, SP.
Ribeiro, P. C. C., Scavarda, A. J., Batalha, M. O., e Bailey, D. (2009), “Application of an IT evaluation method”, International Journal of E-Business Management, Vol. 3, No. 2, pp. 24-42.
Rivard, S.; Huff, S. L. (1984), “User developed applications: evaluation of success from the DP Department Perspective”, MIS Quarterly, Vol. 8, No. 1, pp. 39-50.
Rogers, E. M. (1995), Diffusion of innovations, 4th ed., The Free Press, New York.
Rogers, E. M.; Shoemaker, F. F. (1971), Communication of Innovations: A Cross-Cultural Approach, The Free Press, New York.
Sanchez Limón, M. L., De la Garza Cárdenas, M. H. (2018), “Tecnologías de información y desempeño organizacional de las pymes del noreste de México”, Revista Venezolana de Gerencia (RVG), Vol. 23, No. 82, pp. 298-313.
Sanders, G. L.; Courtney, J. F. A (1985), “Field Study of Organizational Factors Influencing DSS Success”, MIS Quarterly, Vol. 9, No. 1, pp. 77–93.
Schulz, M.; Winter, P.; Choi, S. K. T. (2015), “On the relevance of reports-Integrating an automated archiving component into a business intelligence system”, International Journal of Information Management, Vol. 35, No. 6, pp. 662–671.
Sedera, D.; Gable, G.; Chan, T. (2004), “A factor and structural equation analysis of the enterprise systems success measurement model”, artigo apresentado na AMCIS, Americas Conference on Information Systems, New York, 2004
Srinivasan, A. (1985), “Alternative Measures of System Effectiveness: Associations and Implications” MIS Quarterly, Vol. 9, No. 3, pp. 243–253.
Standing, C., Guilfoyle, A., Lin, C., e Love, P. E. (2006), “The attribution of success and failure in IT projects”, Industrial Management e Data Systems, Vol. 106, No. 8, pp. 1148-1165.
Suh, H., Van Hillegersberg, J., Choi, J., e Chung, S. (2013), “Effects of strategic alignment on IS success: the mediation role of IS investment in Korea”, Information Technology and Management, Vol. 14, No. 1, pp. 7-27.
Swanson, E. B. (1974), “Management Information Systems: Appreciation and Involvement”, Management Science, Vol. 21, No. 2, pp. 178–188.
Venkatesh, V., Morris, M. G., Davis, G. B., e Davis, F. D. (2003), “User acceptance of information technology: Toward a unified view”, MIS Quarterly, Vol. 27, No. 3, pp. 425-478.
Wang, C. H. (2016), “A novel approach to conduct the importance-satisfaction analysis for acquiring typical user groups in business-intelligence systems”, Computers in Human Behavior, Vol. 54, pp. 673–681.
Wang, Y. S.; Liao, Y. W. (2008), “Assessing eGovernment systems success: A validation of the DeLone and McLean model of information systems success”, Government Information Quarterly, Vol. 25, No. 4, pp. 717–733.
Weigel, F. K.; Hazen, B. T. (2014), “Technical proficiency for IS Success”, Computers in Human Behavior, Vol. 31, No. 1, pp. 27–36.
Williams, S.; Williams, N. (2004), “Assessing BI Readiness: The Key to BI ROI”, Business Intelligence Journal, Vol. 9, No. 3, pp. 15–23.
Xu, X.; Wang, L.; Newman, S. T. (2011), “Computer-aided process planning – A critical review of recent developments and future trends”, International Journal of Computer Integrated Manufacturing, Vol. 24, No. 1, pp. 1–31.
Yeoh, W.; Koronios, A. (2010), “Critical Success Factors for Business Intelligence Systems”, Journal of Computer Information Systems, Vol. 50, No. 3, pp. 23–32.
Zha, X., Zhang, J., Yan, Y., e Xiao, Z. (2015), “Does affinity matter? Slow effects of e-quality on information seeking in virtual communities”, Library e Information Science Research, Vol. 37, No. 1, pp. 68-76.
Zmud, R. W.; Boynton, A. C.; Jacobs, G. C. (1987), “An Examination of Managerial Strategies for Increasing Information Technology Penetration in Organizations”, apresentado na ICIS, International Conference on Information Systems, Pensylvannia, Estados Unidos, 1987.
Received: Nov 15 2018
Approved: Jan 07 2019
DOI: 10.20985/1980-5160.2019.v14n1.1480
How to cite: Fernandes, B. B.; Ribeiro, P. C. C.; Costa, H. G. (2019), “Evaluation of Business Intelligence in telecommunications companies”, Sistemas & Gestão, Vol. 14, N. 1, pp. 64-76, available from: http://www.revistasg.uff.br/index.php/sg/article/view/1480 (access day abbreviated month).