Fluminense Federal University – UFF, Niterói, Rio de Janeiro, Brazil.
Fluminense Federal University – UFF, Niterói, Rio de Janeiro, Brazil.
The performance evaluation of the effluent treatment plants (ETP) is generally based on measuring only the quality of the final effluent, not taking into account other parameters, which also contribute substantially to the performance. In this sense, indicators are important tools, for being able to summarize in a single value the state of the object evaluated, considering its various aspects and characteristics. The objective of this work is to score the importance of performance indicators for ETP evaluation and licensing, in addition to evaluating the use of ETP quality indicators established through two institutional norms (NOI-INEA 11 and NOI-INEA 14) prepared by the State Institute of Environment (INEA - Instituto Estadual do Ambiente), an environmental agency of the state of Rio de Janeiro. A bibliographic review was carried out on articles on the subject of ETP performance indicators, together with a survey on the websites of all state and district environmental agencies. In addition, for critical analysis, the methodology for researching official documents in INEA's digital archive was adopted. With this, it was possible to perceive that, in Brazil, the existence of legislation that uses indicators for ETP evaluation is still restricted to the alluded state. However, despite being little used, the indicators have great potential to assist in environmental licensing and in the substantiation of technical opinions, contributing to better information and participation of society.
Descriptors: Environmental licensing; Environmental management; State environmental agencies; Environmental quality.
The main objective of the effluent treatment plants (ETP) is to promote the purification of effluents, whether of domestic or industrial origin, before their final disposal, usually in water bodies. With this, the aim is to implement increasingly efficient technologies for removing the polluting load from the effluent and, if possible, with the generation of little waste and rejects.
However, the need arises to evaluate the performance of these stations, in a more holistic way, beyond the compliance with the launching standards stipulated in the environmental legislation. According to Raschle (2013), performance evaluation is an important tool to identify operation, safety and environmental deficiencies of ETP.
For Piza and Paganini (2006), indicators are management instruments by which the performance and functioning of a service or productive unit are measured, through which it is possible to identify necessary improvements, such as adjustments and/or expansions.
The Water, Energy and Sanitation Regulatory Agency of the Federal District (ADASA, 2016, p. 37) conceptualizes performance indicators as: "indexes that summarize the most relevant aspects of the performance, in general, operational and economic-financial of an entity, simplifying its analysis".
Thus, indicators can be defined as a tool of quantified information with the ability to synthesize and compile a series of characteristics of the environment or phenomenon evaluated. The indicators inform data about the state of the phenomenon or medium, which is easy to understand, allowing its use in decision making and in the transmission of information to the whole society.
Along the same path, the Ministry of the Environment (MMA – Ministério do Meio Ambiente, 2019) defines environmental indicators as selected statistics that represent or summarize some aspects of the state of the environment, natural resources and related human activities. Therefore, the use of an environmental indicator seems to be a promising alternative in the evaluation of ETP performance, considering that the use of environmental performance indicators is already widely carried out in industries around the world (Melo; Souza, 2014). Furthermore, through the results of indicators it is possible to carry out an evaluation of the evolution of performance over time, as well as to comparatively analyze some organizations in the sector, including entities from other regions (ADASA, 2016).
Therefore, the objective of this article is to evaluate the use and relevance of quality indicators of effluent stations established through two institutional norms (NOI-INEA 11 and NOI-INEA 14) elaborated by the State Institute of the Environment (INEA), environmental agency of the state of Rio de Janeiro.
Initially, in the present article, a bibliographic review was carried out in articles related to the theme of effluent treatment, through which three national articles (Sperling; Sperling, 2013; Barros, 2013; de Paula, 2013) and three international articles (Lindtner et al., 2008; Balmér; Hellström, 2012; ERSAR, 2020) were selected as the basis for this study. The survey was conducted from June 2018 to March 2020, using the keywords: performance indicators, sewage treatment plants, quality indicators, and industrial waste treatment plants. The search was carried out in the main journal portals, such as CAPES and SciELO. The selection criterion was the framing of keywords and relevance to this article. In addition, a survey was carried out on the websites of the Brazilian state environmental agencies and the Federal District and in countries of international prominence with scientific production in the area of performance indicators of sewage treatment plants.
In the context of the state of Rio de Janeiro, the methodology for researching official documents (internal regulations, technical opinions, minutes of meetings of the Board of Directors of the Body) was adopted in the digital collection of the state environmental agency – INEA, looking for IQE (Sewage Treatment Plant Operation Quality Index), IQETDI (Industrial Waste Treatment Plant Operation Quality Index), Quality Index of Treatment Plants, in addition to holding meetings with those responsible for the sector of ETP licensing and exchange of e-mails with the main technical creator of the indexes in this body.
As diagnosed by Barros (2013), there are several organizations at the international level that have a system of development indicators on the sanitation system. However, there are few specific indicator systems for sewage treatment plants, in particular the Austrian Water and Waste Association (AWWA), the National Civil Engineering Laboratory of Portugal (LNEC) and the Swedish Water & Wastewater Association (SWWA).
AWWA has developed the Austrian benchmarking system for six years (1999-2004) and its main objectives were the development of performance indicators and the identification of good performances and potential cost reductions of ETP (Lindtner et al., 2008). As noted by de Paula (2013), the AWWA performance indicator is classified into three main categories: financial, technical and process. The system performance is obtained from the correlation between process indicators and technical indicators.
LNEC has developed a project called PASt21 which is a National Initiative for Performance Evaluation of Urban Water Treatment (WTP) and Sewage Treatment Plants (ETP) and aims to establish performance evaluation and benchmarking, and to reinforce the applicability of the performance evaluation system of the mentioned stations, including the different typologies. The performance indicator analyzes eight areas of the station: the quality of treated wastewater; efficiency and reliability; water use, energy and materials; management of by-products; safety; human resources; economic and financial resources; and planning and project support.
The indicator system created by SWWA in 2003 on an internet platform called VASS, with the start of data collection from sewage treatment plants in Sweden in 2009, had the purpose of providing a comparison between the ETPs of the country and other countries, through indicators, enabling benchmarking between the plants. The evaluation of the ETPs involves the following dimensions: effluent quality, sludge, energy, chemical products, and economic aspects (Balmér; Hellström, 2012).
It is important to highlight a common feature in the objectives of the indicator systems mentioned above, which is benchmarking. This practice is possible because most ETPs are public companies, and this allows the sharing of good practices and success cases.
The Basic Sanitation Law, Law no. 11,445/2007 (Brazil, 2007), provides in its article 23 on the minimum aspects of the evaluation of sanitation service provision, of which the following should be highlighted: standards and indicators of quality of service provision; progressive goals of expansion and quality of services and respective deadlines; monitoring of costs; evaluation of efficiency and effectiveness of services provided; standards of care to the public and mechanisms of participation and information.
In this sense, it is imperative that Brazil also has indicators capable of evaluating the ETP, in a more comprehensive way than only in terms of meeting the standards for effluent discharge recommended by legislation, since compliance with environmental legislation may not ensure full protection of the environmental quality of the bodies receiving effluents (Bertoletti, 2015).
In addition, in general there is also a benefit for public health, as currently in Brazil water-binding diseases, such as cholera, typhoid and parathyroid fevers, amebiasis, diarrhea, and schistosomiasis are responsible for a high number of hospitalizations. According to the Trata Brasil Institute (2019), the incidence of hospitalizations for diseases associated with lack of sanitation was 12.46 hospitalizations per 10 thousand inhabitants.
Currently, in the country, there are indicators to evaluate exhaustion, where perhaps the best known is the National Sanitation Information System (SNIS). Such system has national scope and collects institutional, administrative, operational, managerial, economic-financial, accounting and quality data on basic sanitation services in urban areas of the four components of basic sanitation (water, sewage, solid waste and drainage, and rainwater management).
The SNIS has the following objectives: (i) planning and execution of public policies; (ii) orientation of resource application; (iii) knowledge and assessment of the sanitation sector; (iv) assessment of services performance; (v) improvement of management; (vi) orientation of regulatory and inspection activities; and (vii) exercise of social control.
Other organizations also have a proposal of indicators to evaluate the exhaustion system, highlighting the Brazilian Association of Regulatory Agencies (ABAR - Associação Brasileira de Agências de Regulação), the Regulatory Agency of Delegated Public Services of the State of Ceará (ARCE - Agência Reguladora de Serviços Públicos Delegados do Estado do Ceará) and ADASA.
However, these indicator systems are geared towards evaluating the performance of depletion systems in a more global manner, comprising other parts besides the ETP, such as collecting network and customer satisfaction relationship, for example. On this issue, in Brazil, there are few performance indicators that are exclusively dedicated to the evaluation of ETP, and the indicators proposed by Barros (2013) for the evaluation of ETPs in Brasília - DF and the system proposed by de Paula (2013) can be cited.
In this context, it is worth bringing information on the existence of the National Sanitation Quality Award (PNQS - Prêmio Nacional de Qualidade em Saneamento) of the Brazilian Association of Sanitary and Environmental Engineering (ABES - Associação Brasileira de Engenharia Sanitária e Ambiental). Every year, the best performing sanitation service companies are awarded.
The evaluation of the companies is made through performance indicators, published annually in the Reference Guide for Performance Measurement (GRMD - Guia de Referência para Medição do Desempenho) and the list of indicators is divided into five groups: economic-financial, social and environmental, related to clients and market, related to people and related to the process.
In the last editions, the awarded companies have been from the Southeast, including the Basic Sanitation Company of the State of São Paulo (SABESP - Companhia de Saneamento Básico do Estado de São Paulo) and the Sanitation Company of Minas Gerais (COPASA - Companhia de Saneamento de Minas Gerais) can be highlighted. In the other regions of the country, the following companies stand out: Companhia Riograndense de Saneamento (CORSAN) and Companhia de Saneamento do Paraná (SANEPAR); in the Northeast: Companhia de Saneamento de Sergipe (DESO) and Empresa Baiana de Águas e Saneamento (EMBASA); and in the Midwest: Companhia de Saneamento de Goiás (SANEAGO).
In the context of the state of Rio de Janeiro, INEA was created in 2007, with the mission of integrating the state environmental policy, being a reference environmental management agency, playing a strategic role in state development.
In 2015, through the Board of Directors (CONDIR), INEA's highest body, composed of the Institute's Presidents and Directors, two institutional norms (NOI) were approved, NOI-INEA 11 (INEA, 2019) and NOI-INEA 14 (INEA, 2019). These norms aim to standardize and support the methodology for calculating the IQE and the IQETDI in the state of Rio de Janeiro, respectively.
The referred indexes are calculated from a series of items related to the structure, quality of the treated effluent and legal compliance, and for the result of each item a weight is assigned: one weight for a positive result and another for a negative result. In some cases, another weight is assigned for an intermediate result. The results are obtained by means of a percentage evaluation of compliance with the criteria and an index ranging from zero to ten, and they are classified as shown in Table 1.
Table 1. Evaluation of IQE and IQETDI framework
Source: Author (2020)
With this, the environmental agency outlines and differentiates the evaluation of the station operation quality by the predominant characteristic of the effluent to be treated, that is, according to its origin – domestic or industrial.
The creation of an evaluation index of effluent treatment plants places INEA as a pioneer in this matter, since no legislation on this merit has been found in any other state or district environmental agency. Furthermore, not even the federal environmental agency, the Brazilian Institute of Environment and Renewable Natural Resources (IBAMA - Instituto Brasileiro do Meio Ambiente), has legislation on this matter.
Thus, it is clear that INEA has a very important tool to evaluate the stations. However, since the publication of the indexes (IQE and IQETDI), there are few records of their application in technical opinions prepared by the agency, as searched in the institutional website. This fact was confirmed in an interview with the head of service and with the environmental analyst responsible for this area within the agency (Management of Licensing of Non-Industrial Activities - GELANI). It was found that IQE has already been used in a technical opinion rejecting a renewal process of a license to operate a domestic sewage treatment plant of the State Water and Sewage Company of Rio de Janeiro (CEDAE - Companhia Estadual de Águas e Esgoto do Rio de Janeiro) (INEA, 2018).
The indicators have great versatility, as they have the potential to evaluate not only the quality of the stations but also their global efficiency, should energy consumption and input parameters be reviewed and added, for example, and following the models of performance indicators highlighted at the international level, as pointed out in the previous item of this study. This way, these indexes would be able to evaluate the performance of two stations equally and make comparison possible. This characteristic is even more interesting from the point of view of entrepreneurs, who will be able to evaluate their station not only from the environmental perspective, but also from the operational and economic efficiency perspective, taking into account cost-benefit criteria.
A relevant point to be highlighted from these standards is the forecast of the annual disclosure of the indicators on INEA's Intranet. It is interesting to note that this feature dialogues directly with the principle of information, one of the principles of environmental law.
The main objective of the information principle in environmental law is, in effecting the right to information, to allow individuals to participate actively in issues related to the environment. [...] And this participation can take place both in the private or individual context, with the intention of reducing environmental degradation, as well as in the public sphere, imposing on the administrative and judicial authorities an adequate and effective action, through the legal means available (Gomes; Simioni, 2014, p. 129).
The disclosure of results to the population is also related to the fundamental principle of transparency of actions - principle IX - established in Law No. 11,445/2007, which establishes the fundamental principles for public services of basic sanitation. Another point that is important to mention is the provision in article 23, X of the Law, which deals with the need for providing information on sanitation, namely: "standards of public attendance and mechanisms of participation and information" (emphasis added).
On the other hand, contrary to what INEA does with other indexes, such as the Water Quality Index, which evaluates the quality of rivers, there is no periodic disclosure of the results of ETP. In fact, there are no records in the database of INEA's website of any publication of the results related to the indices of the state's effluent treatment plants.
A comparison was made between the indexes created by INEA, whose summary is shown in Table 2.
Table 2. Comparison between IQE and IQETDI criteria
Source: Author (2020)
By comparing the two indicators, IQE and IQETDI, it is generally noted that IQE takes into account more criteria than IQETDI. Furthermore, the criteria linked to the location of the station, the contingency plan and the maintenance status of the equipment are evaluated only in IQE. However, this difference in the evaluation is questionable, since these criteria are not related to the origin of the effluent or to any specific domestic ETP legislation.
There is also a significant difference in the weights attributed to the criteria in common between the indices. The criterion that evaluates the number of violations committed in the IQETDI is twice the weight of the IQE; the same occurs in the criterion on access to the flow meter and proper disposal of waste.
Such differences are not justified because the legal instruments regulating these items do not back them up. For instance, the inadequate destination of waste is penalized by State Law No. 3467/2000 (Rio de Janeiro, 2000), without distinction on its origin (industrial or domestic).
Although both indexes aim to evaluate the performance of ETP, there is a distinction regarding the typology of the treated effluent (domestic or industrial). Nonetheless, the differences found between the two indices are not related to the type of effluent to be treated, i.e., they do not configure a different evaluation or approach. The difference is concentrated only in the quantity and the weights assigned in some criteria.
The criteria that are evaluated only in IQE are not particular to domestic effluent stations; on the contrary, they could also be evaluated in industrial effluent stations. Among these criteria, the presence of a disinfection system and the sedimentable and floating materials in the treated effluent can be mentioned.
It is worth pointing out an error in the evaluation of the characteristic odor parameter in the effluent, in both indexes, because the score is given when there is presence of characteristic odor. However, it makes no sense to penalize the odorless effluent and to bonus the effluent that presents odor.
Thus, the objective of the agency in instituting two distinct indicators is unclear. It would have been possible to create only one that evaluated the ETP, whether of domestic or industrial origin. Moreover, as perceived by the analysis, the difference in the evaluation of the treatment station regarding the type of effluent is not found in the indicators proposed in other countries (LNEC, SWWA and AWWA, for example). Thus, a single indicator evaluating ETP could be created, encompassing not only the criteria set out in the two current indices, but also criteria capable of evaluating energy efficiency and input management.
Another significant point, besides the revision of the criteria, would be the revision of the weights applied to them, and research could be carried out with important players in the area (analysts from the agency, academic institutions and companies in the field), as proposed in the studies conducted by Sperling and Sperling (2013), Barros (2013), and Paula (2013).
The state of Rio de Janeiro made public its new State Environmental Licensing and Control System (SELCA) through the publication of State Decree No. 46,890 of December 23, 2019, which revoked the old Environmental Licensing System (SLAM - Sistema de Licenciamento Ambiental). The current system has gone through public consultation and had as main bias the simplification of the environmental licensing procedure, in consonance with what was recommended in the last version of Bill No. 3,729/2004 (Brazil, 2004), authored by Federal Congressman Kim Kataguiri, in progress in the Federal Congress.
The Decree No. 46,890/2019, in its article 13, establishes:
The licensing and other environmental control procedures will take into account performance indicators of the enterprise or activity (...) with a view to effectiveness in the protection of the environment ecologically balanced and the economic and social development of the state of Rio de Janeiro. (emphasis added).
The normative text expresses the importance of performance indicators in the evaluation and environmental control of enterprises and activities, especially in the area of environmental licensing. In this way, SELCA places responsibility for establishing indicators for the state environmental agency itself, according to article 14: "The competent environmental agency will seek to establish, as a general rule, the adoption of performance indicators, instead of means to reach them, in respect to the principle of free initiative".
At this juncture, it is noted the need for INEA to reassess and promote the application of its ETP, IQE and IQETDI quality assessment indices, since these have acquired greater legal importance with the publication of SELCA in the state of Rio de Janeiro.
Another interesting point that emerges with the new state environmental licensing system is the dependence on the evaluation of sustainability criteria for setting maximum and minimum times for environmental licenses. These criteria are divided into four groups: environmental management; products and waste; water efficiency, energy and emissions; and nature conservation.
The ETP performance indicator is an excellent tool capable of evaluating the efficiency of the management of inputs and energy. Likewise, it is worth mentioning that the indicator enables the continuous improvement of the station, considering that in its application points of improvement and correction are identified (ADASA, 2016; ERSAR, 2020).
For entrepreneurs, the implementation of this indicator is attractive, since they can contribute so that the validity of the environmental license of their activity reaches the maximum term established in the aforementioned decree. It is worth noting that in relation to the operating license (OL), for example, the difference between the minimum and maximum terms is six years. The same happens with the unified environmental license (LAU), whose minimum term is six years and the maximum is 12 years, that is, a difference of six years as well.
Still on the aspect of the renewal of environmental licenses, the Decree states, in its Article 31, that "in cases where the Environmental Control Audit Report is approved by INEA, without detecting non-conformities, the renewal of the Environmental License may be carried out expeditiously, as provided for by regulation".
As a result, the Environmental Control Audit, regulated by INEA Guideline 056-R3 (INEA, 2010), gains greater prominence in environmental licensing, becoming a timely opportunity for auditors to apply and evaluate IQE or IQETDI in order to support and substantiate, in an analytical manner, the final report of the audit. It is worth pointing out that one of the major obstacles faced by the environmental agency in applying these indicators is the reduced number of technicians, according to the statement provided in an interview by the analyst responsible for the ETP licensing service at INEA. Therefore, the indicator would be applied at least once a year in the effluent treatment plants of enterprises inspected and licensed by the state.
The Project of Law n° 3729/2004 that promulgates the General Law of the Environmental Licensing, in its last text in progress in the Congress, establishes in article 15 that enterprises that have adopted measures that allow reaching more rigorous results than the criteria established by the environmental legislation in force may enjoy special conditions, such as prioritization in the analysis and expansion of the periods of renewal of environmental licenses. Therefore, the implementation of performance indicators by entrepreneurs is in line with the current text of this bill.
Environmental performance indicators are instruments that contribute to environmental preservation, since they favor a more careful evaluation and, in general, stimulate the search for higher level results. These instruments are in accordance with the fundamental right advocated in article 225 of the 1988 Federal Constitution, which states that every individual has the right to an ecologically balanced environment.
In relation to the ETP evaluation, the performance indicators can offer an extremely relevant contribution as evaluation criteria, benchmark of good practices, grounding in technical opinions from environmental agencies and also in Environmental Licensing, as the environmental legislation of the state of Rio de Janeiro and in progress in the National Congress have pointed out.
In this context, INEA presented ETP performance indicators that have a high potential to apply and obtain information and results regarding stations in a clear and objective way, and can be disseminated to the general population, facilitating access to information that is too technical and difficult to understand by lay people.
On the other hand, there are still improvements, such as the unification of the two indicators (IQE and IQETDI) generating a single performance indicator for the evaluation of ETP (domestic or industrial). Another important point is that this single indicator provides evaluation of ETP in the following dimensions: operation and infrastructure, quality (effluent and sludge), management of inputs and economic-financial aspects, as observed in international performance indicators. On the other hand, there are still improvements, such as the unification of the two indicators (IQE and IQETDI) generating a single performance indicator for the evaluation of ETP (domestic or industrial). Another important point is that this single indicator provides evaluation of ETP in the following dimensions: operation and infrastructure, quality (effluent and sludge), management of inputs and economic-financial aspects, as observed in international performance indicators.
Agência Reguladora de Águas, Energia e Saneamento Básico do Distrito Federal – ADASA (2016). Resolução nº 008, de 04 de julho de 2016. Diário Oficial do Distrito Federal, 128, 6 jul. 2016.
Balmér, P.; Hellström, D. (2012), Performance indicators for wastewater treatment plants. Water Science and Technology 65, 7, 1304-1310. https://doi.org/10.2166/wst.2012.014
Barros, I. P. A. F. (2013). Proposta de um sistema de indicadores de desempenho para avaliação de Estações de Tratamento de Esgotos do Distrito Federal. Dissertação de Mestrado, Universidade Federal de Minas Gerais, Minas Gerais. Disponível em: www.smarh.eng.ufmg.br/defesas/804M.PDF (acesso em: 10 ago. 2019).
Bertoletti, E. (2015). A Presunção Ambiental e a Ecotoxicologia Aquática, Ministério Público Federal, disponível em: www.mpf.mp.br/atuacao-tematica/ccr4/dados-da-atuacao/projetos/qualidade-da-agua/boletim-das-aguas/edicao-2015/a-presuncao-ambiental-e-a-ecotoxicologia-aquatica-1 (acesso em: 26 abr. 2020).
Brasil. Congresso Nacional. Subemenda substitutiva global de plenário. Projeto de Lei da Câmara n.º 3.729 de 2004. Autoria: Deputado Kim Kataguiri (2019), disponível em: www2.camara.leg.br/atividade-legislativa/comissoes/grupos-de-trabalho/56a-legislatura/licenciamento-ambiental/documentos/outros-documentos/texto-base-4a-versao-apresentado-em-08-08.2019 (acesso em: 21 mar. 2020).
Brasil. Lei 11.445, de 5 de janeiro de 2007. Diário Oficial da União, 8 jan. 2007. Disponível em: www.planalto.gov.br/ccivil_03/_ato2007-2010/2007/lei/l11445.htm (acesso em: 21 mar. 2020).
De Paula, R. L. (2013). Metodologia para avaliação de desempenho operacional de estações de tratamento de esgotos, utilizando métodos multiobjetivo e indicadores. Dissertação de Mestrado, Universidade de Brasília, Brasília, DF, Brasil. Disponível: https://repositorio.unb.br/bitstream/10482/14718/1/2013_ReuelLopesDePaula.pdf. (acesso em: 23 mar. 2020).
Entidade Reguladora de Águas e Resíduos de Portugal – ERSAR (2020). Guia Técnico 22 - Guia de Avaliação da Qualidade dos Serviços de Águas e Resíduos Prestados aos Utilizadores - 3ª Geração do Sistema de Avaliação, Lisboa, 2020. Disponível em: www.ersar.pt/pt/publicacoes/publicacoes-tecnicas/guias (acesso em: 23 mar. 2020).
Gomes, R. N.; Simioni, R. L. (2014). A aplicação do princípio ambiental da informação no direito brasileiro na forma de confiança e risco em Niklas Luhmann. Revista Direito Ambiental e Sociedade 4, 2, 117-136. Disponível em: www.ucs.br/etc/revistas/index.php/direitoambiental/article/view/3341/2255 (acesso em: 10 ago. 2019).
Instituto Estadual do Ambiente – INEA (2010). Diretriz para Realização de Auditoria Ambiental. Diretriz INEA 056-R3. Rio de Janeiro, RJ. Disponível em: http://www.inea.rj.gov.br/wp-content/uploads/2019/11/DZ-0056.R-3.pdf. (acesso em: 10/03/2020)
Instituto Estadual do Ambiente – INEA (2015). Metodologia para cálculo do Índice de Qualidade de Operação de Estação de Tratamento de Esgoto (IQE), NOI/INEA-11, Rio de Janeiro, RJ.
Instituto Estadual do Ambiente – INEA (2015). Metodologia para cálculo do Índice de Qualidade de Operação de Estação de Tratamento de Despejos Industriais (IQETDI), NOI/INEA-14, Rio de Janeiro, RJ.
Instituto Estadual do Ambiente – INEA (2018). ATA da 425ª Reunião Ordinária de Licenciamento Ambiental do Condir do dia 04/04/2018. Disponível em: www.inea.rj.gov.br/cs/groups/public/@inter_pres_aspres/documents/document/zwew/mtu0/~edisp/inea0154207.pdf (acesso em: 10 ago. 2019).
Instituto Estadual do Ambiente – INEA (2018). Parecer Técnico de Licença de Operação n° 13/18 fl 1/7, Sistema de Consulta Unificada de Processo do INEA. Disponível em: http://sistemas.inea.rj.gov.br/meioambiente/arquivos/licenciamento/parecer/164636/PARECER%20GELANI%20N%C2%B0%2013-18%20-INDEF%20LO%20IN001546-ETIG%20CEDAE..pdf (acesso em: 10 ago. 2019).
Instituto Trata Brasil (2019). Internações de doenças por veiculação hídrica no Brasil. Disponível em: www.tratabrasil.org.br/blog/2019/05/21/internacoes-de-doencas-por-veiculacao-hidrica-no-brasil/ (acesso em: 10 ago. 2019).
Lindtner, S., Schaar, H., Kroiss, H. (2008). Benchmarking of large municipal wastewater treatment plants treating over 100,000 PE in Austria. Water Science and Technology 50, 7, 1487-1493.
Melo, J. F. M.; Sousa, A. F. (2014). Indicadores de desempenho ambiental no setor público: uma análise nos gastos ambientais e no desempenho verde de municípios, artigo apresentado nos Anais do XVI Encontro Internacional sobre Gestão Empresarial e Meio Ambiente - ENGEMA, São Paulo, SP, Brasil, 2014. Disponível em: www.engema.org.br/XVIENGEMA/18.pdf, (acesso em: 21 mar. 2020).
Ministério do Meio Ambiente – MMA. (2019). Indicadores Ambientais. Disponível em: www.mma.gov.br/informacoes-ambientais/indicadores-ambientais (acesso em: 10 ago. 2019).
Piza, F.J.T.; Paganini, W.S. (2006). Uma proposta de indicadores. In Galvão Júnior, A.C.; Silva, A.C. Regulação: indicadores para a prestação de serviços de água e esgoto. 2 ed. Fortaleza: Expressão Gráfica. p. 204.
Raschle, L. H. (2013). Avaliação de desempenho para estações de tratamento de efluentes industriais. Dissertação de mestrado, Universidade do Estado do Rio de Janeiro, Rio de Janeiro, RJ. Disponível em: www.peamb.eng.uerj.br/trabalhosconclusao/2013/Lilian-Dissertacao.pdf (acesso em: 21 mar. 2020).
Rio de Janeiro (2000). Lei nº 3.467, de 14 de setembro de 2000. Diário Oficial do Estado do Rio de Janeiro, Rio de Janeiro, 15 set. 2000.
Rio de Janeiro (2019). Decreto nº 46.890, de 23 de dezembro de 2019. Diário Oficial do Estado do Rio de Janeiro, Rio de Janeiro, 24 dez. 2019, Seção 1, p. 5. Sperling, T. L. V.; Sperling, M. V. (2013). Proposição de um sistema de indicadores de desempenho para avaliação da qualidade dos serviços de esgotamento sanitário. Engenharia Sanitária e Ambiental 18, 4, 313-322.
Received: Jul 02, 2020
Approved: 17 ago. 2020
DOI: 10.20985/1980-5160.2020.v15n2.1659
How to cite: Ramos Junior, W. (2020). Critical analysis of the performance indicators of effluent treatment plants of the state environmental agency of Rio de Janeiro. Revista S&G 15, 2, 181-189. https://revistasg.emnuvens.com.br/sg/article/view/1659