PMAQ-CEO External Evaluation : Revealing Indicators for Planning Services

Objective: To analyze the indicators of planning and management actions for the work process organization in Centers of Dental Specialties (CEO) presented in the external evaluation tool to the Improving Access and Quality of CEO AIQP-CEO Program and to provide a follow-up and monitoring tool of goals. Material and Methods: Crosssectional study with the use of secondary data from the external evaluation of 930 CEOs in Brazil. A descriptive analyses of indicators identified in the “Planning and managing actions for work process organization in CEO” sub-dimension was performed. It provides two monitoring tools: monthly production for each specialty, and monthly minimal and mandatory performance of CEO goals. Results: It was shown that 85.4% of CEO teams have made the monitoring of the goals established to each offered specialty, and 73.1% have performed self-assessment in the last six months, in which most of the teams (647 69.6%) have used the Self-Assessment to the Improving Access and Quality Program SAIQP. It was demonstrated that of the 680 teams that performed selfassessment, 612 (65.8%) consider the results in the teamwork organization. Conclusion: The PMAQ CEO external evaluation brought important information to the qualification process of health services. Concerning the planning, the majority of teams performed the follow-up of goals and self-assessment, and they used the self-assessment results in the teamwork organization. According to results, some tools should be used to support CEO teams in the follow-up of the monthly production by specialty and in the monitoring of goals.


Introduction
Planning is a tool of excellence and powerful for qualification of the Unified Health System (SUS).Thus, there is evidence of incentives and initiatives aimed at retrieving or building the planning culture in SUS [1].
The existing decentralization in SUS provided more precise diagnoses of the local reality and the identification of the health determinants involved, offering the possibility of adequate management aimed at the needs of the population.This gives municipalities more power and responsibility, which makes planning essential and situational, as opposed to the centralized and normative planning of public management until 1990.However, due to the demands required in the daily work, this practice is sometimes neglected and actions are improvised to solve emerging problems [2].
The function of planning -including monitoring and evaluation -is inherent to all professionals, regardless of where they work.Planning has enormous potential to enable the achievement of timely resolubility of the established and implemented measures.This is one of the main potentialities that give planning the strategic character for management qualification [1] (p.9).
Monitoring is understood as the continuous monitoring of commitments -targets, goals and actions -arranged in projects, schedules or plans, in order to verify if they are being executed as recommended.Evaluation is understood as a process of judgment in which what has been accomplished is analyzed -intervention, action, service, procedure -or the result obtained, comparing with the ideal to be reached [1].
In this way, both terms have evaluative character, that is, identify fragile points and potentialities, so that measures can be implemented to overcome the challenges that impair the progress or reaching the proposed results.
The systematic practice of monitoring and evaluation in health services should be implemented.It is important to use data and information as "part of the work process and as a tool to improve the offer of services to the population" with the establishment of strategic planning [3].

It is essential to spend time to propose actions, goals and analyze their compliance.
Implementing the monitoring process in health actions is a fundamental task to monitor the performance of services as well as for decision making, whether in primary, secondary or tertiary care.
"The act of evaluating adds value when the knowledge and use of the information produced generates institutional and professional improvement" [4].For this author, the evaluation and monitoring process has impact on several dimensions: management, care and epidemiological profile.Thus, the implementation of systematic evaluations has been stimulated by the interest in associating investments in the health sector with the results obtained, since available resources are scarce [5].
In the analysis of indicators used in oral health in Brazil present in government guidelines and scientific articles published between 2000 and 2012, it has been observed that over time the use of indicators has been widely expanded, however, in oral health, practice has shown that there is still much to be done so that managers and health professionals make use of this tool [6].
The indicators proposed by managers for the monitoring and evaluation of oral health care, as recommended by the Ministry of Health, mostly refer to primary care and have proved to be insufficient for evaluation and decision-making.The need to use different health indicators that allow the evaluation of actions and that can be incorporated into the work routine is highlighted [6].

Study Design
A cross-sectional study was carried out with secondary data from the external evaluation of 930 Centers of Dental Specialties in Brazil.The edition of Ordinance No. 261, of February 21, 2013 [9], established the PMAQ-CEO in the scope of the National Oral Health Policy, and later, Ordinance No. 1599 was published in 2015, which applies on this program [10].
The AVE PMAQ-CEO was composed of the following aspects: observation of the infrastructure conditions of Health Units; services provided by the Health Unit from the point of view of the manager and health professionals and analysis of users' satisfaction and their perception regarding health services, their access and use.
Thus, the AVE PMAQ-CEO instrument evaluates services in different dimensions and subdimensions and presents different indicators in their sub-dimensions.
A descriptive analysis of indicators identified in the "Planning and management actions to organize the CEO's work process" sub-dimension, arranged in the AVE PMAQ-CEO, was performed and these indicators were classified into categories (Table 1).

Table 1. "Planning and management actions to organize the CEO's work process" sub-dimension, arranged in PMAQ-CEO external evaluation. Sub-Dimension Category
Planning and management actions for organization of the CEO's work process Monitoring and analysis of the goals by specialty Team self-assessment Application of the self-assessment result in the organization of the work process After analyzing the indicators presented in the selected sub-dimension, a documentary analysis of ordinances issued on CEOs regarding the goals of procedures by specialty was carried out.From these, two monitoring instruments were developed: one to monitor monthly production in each specialty and another instrument to evaluate the CEO's monthly performance regarding the achievement of minimum and mandatory targets.

Ethical Aspects
The PMAQ-CEO evaluation had favorable opinion (Protocol No. 740.874) from the Research Ethics Committee of the Federal University of Pernambuco in compliance with Resolution 466/2012.

Results
The results of indicators presented in the AVE PMAQ-CEO held in 930 CEOs on the planning and management actions for organization of the work process in the category monitoring of goals are described in Table 1.It was observed that the majority (85.4%) of CEO's teams (n = 794) reported that they monitored the goals established for each specialty offered, and that of these, 649 (69.8%) presented a document proving the monitoring of goals.Indicators in the self-assessment category were also identified in the PMAQ CEO instrument (Table 2).The use of the self-assessment result in the organization of the team work process was also questioned.It was verified that of the 680 teams that make self-assessment, 612 consider the results in the teamwork organization, therefore, 65.8% of them.

Discussion
In the scope of public health, the population's access to actions and services of assistance, prevention and promotion of oral health was conquered gradually over time.Comprehensive oral health care ceased to be the exclusive privilege of schoolchildren [11] and became the universal right of all age groups.For a long time, adults were given access to free-demand actions [12], where dental mutilation or pain relief was performed.
In 2004, the exclusion of the majority of the population to oral health services was described as follows: The attention and dental care provided by the public sector in Brazil prioritized almost exclusively students from six to 12 years of age, except for some isolated municipal experiences where health care was directed to other age groups of the population.Punctual and mostly focused on reparative or mutilation assistance offered to the adult population throughout the history of public policies in Brazil did not undergo significant changes after the implementation of the Unified Health System (SUS) in 1990.Among others aspects, it establishes the principles of universality, completeness and equity [13].(p 454) The National Oral Health Policy came to break with this hegemonic logic.In addition to the expansion of primary care actions, it has proposed actions that guarantee full access to oral health for the entire population [14].One of the strategies of organization of the specialized care in dentistry proposed in this Policy was the implantation of CEOs and Regional Laboratories of Dental Prostheses (LRPD).
With the institution of the PMAQ-CEO, the culture of evaluation in these centers of specialties is institutionalized [15].The results showed that there is a concern between professionals and management in monitoring the CEO's performance, allowing the analysis of the results and, consequently, identifying and implementing intervention strategies for coping and correcting directions.Failure to meet goals can cause harm to both users and management.In this way, the monitoring and analysis of actions and results achieved contributes to the improvement of the work process and qualification of the services provided, as well as satisfaction of professionals and users [15].The transfer of resources related to the monthly incentives of Centers of Dental Specialties -CEO will be fully suspended when the minimum monthly production in any of the above mentioned specialties is not reached for two consecutive months or three alternate months in a period of one year, and will be maintained until regularization of the minimum monthly production [19].
A considerable number of teams have the practice of performing self-assessment (Table 2), which is a fundamental strategy to identify potentialities and challenges for the organization of the work process and qualification of actions.This is a reflection of the stimulus to the culture of health planning in the SUS, especially in the care area.Health professionals and management are attentive and adhering to the process of evaluating services.
It is important to emphasize that management must dialogue with teams to raise awareness of the practice of goal monitoring and self-assessment.It was also verified that in addition to AMAQ, there were initiatives with the elaboration of other self-assessment instruments, demonstrating the initiative of the management in establishing this practice.However, these instruments should be socialized to the other CEOs.
Self-Assessment for Improving Access and Quality of Centers of Dental Specialties (AMAQ -CEO) is one of the PMAQ tools used in the process of continuous improvement of access and quality of specialized oral health services throughout the country, collaborating with the reorganization of services and management [15].
The AMAQ-CEO makes it possible to identify and recognize the positive and critical aspects of the CEO's work and health care teams [15]."It is at this moment that individuals and groups involved advance in self-evaluation, self-management, problem identification, and formulation of intervention strategies for the improvement of services, relationships and the work process" (p.10) 15].Thus, self-assessment is an essential attribute to be instituted in the SUS.
The moments of construction or preparation for the implementation of self-assessment processes reach their transformative potential when they are guided by participatory methods, using approaches that promote creativity, which considers the plurality of the actors present, promoting a privileged space for the construction of thought.It is at this moment that individuals produce meanings with the potential to mobilize initiatives for the improvement of services [15].
Most teams that perform self-assessment claim to apply it as a tool for planning and promoting change, which is very positive.Self-assessment represents a strategy for decision-making and central action to improve the quality of health actions "as long as it is meaningful and capable of mobilizing the actors for change and improvement of services" [15].
The fact that some teams claim not to use self-assessment (Table 2) can be motivated by performing it only as a management requirement or because the process does not allow the active participation of actors and consequently does not stimulate reflection on the identified aspects.
In order to promote change, the organization of self-assessment moments should induce "active attitudes of the actors involved in the issues raised, providing participants with adequate moments of reflection, discussion about their practices and possibilities for change" [15].

Monitoring Instruments for CEO's Indicators
In order to monitor and analyze the established goals for each specialty offered by CEOs, follow-up tools are needed for the procedures performed.The performance of CEOs throughout the country and registered by SIA / SUS in 2007 was analyzed and the production of dental procedures was obtained from secondary data and questionnaires addressed to CEO's professional team of 10% of services implemented.Among CEOs analyzed, it was observed that in most of the regions, there was difficulty in meeting the goals related to the monthly production to be reported in the SIA / SUS [20].The authors proposed, among other things, that there should be "definitions of new standards and meeting targets for evaluation and monitoring of these services".Some authors prepared three indicators, based on Ministerial Ordinance No. 600 of 2006 [18], to evaluate the performance of Brazilian CEOs (Chart 1).
These indicators were identified by a survey conducted on indicators used in oral health in Brazil between 2000 and 2012 [6].The indicators constructed based on Ordinance No. 600 [18], constitute a practical tool and easy to apply to analyze the performance of CEOs.
Chart 1. Indicators of the Global Compliance of CEO Goals and their calculation methods.

Form of Calculation INDICATOR: Global CEO's Goals Achievement
Quotient resulting from the monthly average of procedures performed in each subgroup of dental specialties by the number of procedures corresponding to the goal of this subgroup multiplied by 100.It is considered: Target reached -percentage equal to or greater than 100% of the target set for each subgroup.Categorized in: Very poor performance -CEO that did not meet any goals Poor performance -CEO that met only 1 goal Regular performance -CEO that met 2 goals Good performance -CEO that met 3 goals Optimum performance -CEO that met all goals.

INDICATOR: Compliance with Secondary Care in Oral Health
Transformation of the 'Global Compliance of Targets' indicator into a binary variable, categorizing services that: Fulfilled: had at least one goal of the Subgroup of Specialties reached (Periodontics, Endodontics or Minor Oral Surgery); or Failed: no goal was achieved or only the Goal of the Basic Care Subgroup.To monitor the CEO's performance, Ordinance no.1464/2011 [19] defines the minimum procedures / month target that each specialty must perform (Table 3).To monitor the CEO's performance, Ordinance no.1464/2011 [19] defines the minimum goal of procedures / month that each specialty must perform (Chart 2).The Annex of Ordinance No.

INDICATOR: Achievement of the
1464 [19] states that the basic procedures to be performed in any of the three types of CEO are unique to the care of patients with special needs.This establishes that 50% of the minimum production of this specialty must be mandatory restorative procedures as follows: restoration in better organization of actions, expansion of users' access to specialized services in Oral Health and effective integration of CEOs in the health care network.

Conclusion
The PMAQ CEO external evaluation brought important information to the service qualification process.In terms of planning, most teams follow the goals, self-assessment, and use the results of self-assessment in the work process organization.Based on the results, instruments were proposed to support the teams of CEOs to follow the monthly production by specialty and to monitor the goals, with the main objective of subsidizing the planning.
Secondary oral health care in SUS is performed through the National Oral Health Policy, Brasil Sorridente, and the establishment of Centers of Dental Specialties (CEO).In 2011, Ordinance No. 1654 established the Program for Improving Access and Quality of Basic Care (PMAQ-AB) under SUS.In 2015, this Ordinance was replaced by Ordinance No. 1645 / 2015.In continuity with the quality-inducing strategies in the Ministry of Health, in 2013, the Program for Improving Access and Quality of Centers of Dental Specialties (PMAQ-CEO) was established as a strategy for the qualification of specialized oral health services (Ordinance No. 261 / 2013) [7-9].The external evaluation tool of PMAQ-CEO (AVE PMAQ-CEO) brings important oral health indicators to analyze specialized care and subsidize their planning.The aim of the present study was to analyze the planning indicators and actions of the management to organize the work process of CEOs, as well as to present an instrument for follow-up and monitoring the monthly production goals of CEOs.
Basic Attention goals, Endodontics, Periodontics and Surgery Goal compliance by Subgroups of dental procedures: Basic Care (Individual Preventive Procedures, Basic Dentistry and Basic Surgical Dentistry); Periodontics; Endodontics; and Minor Oral Surgery [Surgical Dentistry, Buccomaxillofacial Traumatology and Skin Procedures / Surgeries, Subcutaneous Tissue and Mucosa]

Table 1 . Results of Indicators of the
'Planning and management actions for organization of the CEO's work process' sub-dimension in the category 'monitoring of goals' in the PMAQ-CEO External Evaluation.Number of CEOs with document that demonstrates the monitoring and analysis of goals established for each specialty offered 649 (69.8) 145 (15.6) 136 (14.6)NA= Not applicable.

Table 2 . Results of Indicators of the 'Planning and management actions for organization of the CEO's work process' sub-dimension in the self-assessment category in the PMAQ-CEO External Evaluation.
It was verified that 73.1% (n = 680) of CEO's teams performed self-assessment in the last six months, in which the majority (n = 647) used AMAQ as an instrument, thus 69.6% of the teams.It is noteworthy that 66.7% of teams presented documents proving the use of this instrument.The other teams, in minimal percentages, reported that they performed self-assessment using instruments developed by the municipality, government or other instruments and of these, most of them presented documents proving the use.
Number of CEO teams performing self-assessment with an instrument developed by the municipality 111 (11.9) 569 (61.2) 250 (26.9)Number of CEO teams that have a document proving the performance of self-assessment with an instrument developed by the municipality 96 (10.3) 15 (1.6) 819 (88.1)Number of CEO teams performing self-assessment with an instrument developed by the government is noteworthy that the achievement of goals involves different factors such as the provision of continuous inputs, adequate equipment, adequate instruments, organization of the work process, efficient management of people, and absenteeism of users among others.
[19], defines the criteria for implementation, determines the minimum specialties to be offered and the type of modality of CEO, as well as other provisions.According to Ordinance No. 599[16],CEOs must offer at least the following clinical areas (specialties): oral diagnosis, with emphasis on diagnosis and detection of oral cancer; specialized periodontics; minor oral surgery of soft and hard tissues; endodontics; and care for patients with special needs.Ordinance No. 600 (03/23/2006)[18]deals with the financing of CEOs.This was amended by Ordinance No. 1464 (06/24/2011)[19], which defines the list of procedures that will be considered in the monitoring of CEOs.Monitoring consists of the analysis of a monthly minimum production, to be performed in CEOs, verified through the SUS Outpatient Information Systems (SIA / SUS).According to Ordinance No. 1464, the transfer of financial resources to the CEO by the Ministry of Health is conditional on the fulfillment of goals, and this only occurs if all the clinical areas or specialties previously mentioned reach the monthly goal in relation to the list of recommended procedures.Ordinance No. 1464 presents in its annex, in the 4th paragraph, the following determination:

Table 3 . Demonstration of minimum procedures / month by specialty, described in Ordinance no. 1464/2011, according to type of CEO. CEO Type Minimum Goals of Procedures / Month by Specialty
*Basic Procedures per month.