IMPLEMENTATION OF THE KIRKPATRICK EVALUATION MODEL IN BALAI DIKLAT KEAGAMAAN BANJARMASIN

Nowadays government agencies are required to improve the quality of their services to the community. One of the steps taken to improve that is through training activities. In order to achieve the expected goals, the training that is held must go well, from planning to evaluation of results. Therefore, the evaluation must be done seriously and use the right method or model. The training evaluation model that has been applied at the BDK Banjarmasin refers to the four-level evaluation model or what is often called the Kirkpatrick evaluation model. However, based on initial observation, most of the employees and even some trainers do not know about the Kirkpatrick's evaluation model itself. This can be seen from the number of employees and coaches who do not understand how to do the evaluation, or how to analyze the results. Observations made show that the application of Kirkpatrick's evaluation model in the training at the BDK Banjarmasin was still not effective, because most only focus on levels 1, 2 and 3 while level 4 are still rarely done.


INTRODUCTION
Recently government institution required to improve the quality of its public service. This will improve, among others, by improving the quality of human resources service providers. One of the common measures undertaken to improve the quality of human resources is through training activities. Training activities provided are expected to fill the gap or the gap between the competencies now owned an employee with competencies expected in Office so that it can improve efficiency and effectiveness in the work. Training was an attempt to ease employees learn the skills needed in the job (Noe, 2010) In order to achieve the goals expected, which is to enhance employees' skills, training organized should run well. Training activities provided or implemented can run effectively then it should pay attention to the following things: training needs analysis to set goals to determine the content or materials or to assign the appropriate participants; set a schedule; select an adequate facilities; choose competent instructors; selecting and preparing the supporting equipment; coordinate programs; and evaluate program. However, the actual evaluation is an important part of a series of training activities and the development of human resources. Through evaluation can be known the extent to which these activities bring the impact to employees who become participants and also for the organization or the Office. This is particularly important given the costs incurred for organizing a training is not a little. In fact, at some cost to the Organization of training activities is categorized as an investment that will greatly affect the viability of the organization. Therefore, the evaluation should be done seriously and use appropriate methods or models (Noe, R. A., Hollenbeck, J. R., Gerhart, B., & Wright, 2013;Noe, 2010) Training evaluation model that is applied in Balai Diklat Keagamaan Banjarmasin (BDK Banjarmasin) refers to the four-level evaluation model or often called the Kirkpatrick evaluation model. Nevertheless, because there are still employees who do not know about the model itself then the Kirkpatrick evaluation implementation in the field likely has not been effective. This is unfortunate because the understanding of the model of this evaluation can be useful to improve the application of this model in the field. Therefore, this study is directed to find out what is the four-level evaluation model according to Kirkpatrick and how effective was the application of the four-level model of evaluation according to Kirkpatrick in BDK Banjarmasin?

METHOD
The preparation of this paper was done by using several methods i.e., observation, interview, and also a study of the literature. Observations and interviews were conducted to obtain information about how the condition and existing problems in the field. As for the study of the literature was conducted to get a reference to the form of the theory as well as the results of the study and research as well as the framework of theory and also enrich the discussion. The discussion in this paper is expected to add insight to readers, particularly about the training evaluation model according to Kirkpatrick. In addition, as well as input for the religious Training of Banjarmasin Hall management that ultimately can be used later as a material consideration in the process of the evaluation of training activities.

RESULTS AND DISCUSSION
There are at least three things that underlie training providers to conduct evaluations, namely: to ensure that training programs are developed and presented in ways that maximize learning; to ensure that what is learned is relevant and can be applied in the workplace; to show the value or benefits of training for achieving organizational goals (James D. Kirkpatrick, 2016) Given the magnitude of the benefits that can be gained from the evaluation process, it is important for the organizers to ascertain what criteria or criteria will be used to assess the training activities. Basically, there are two types of evaluations used in evaluating training activities namely formative evaluation and summative evaluation. Formative evaluation is an evaluation carried out during the activity while summative evaluation is carried out after the activity is completed. Information obtained from this process includes the extent to which participants enjoy activities and can learn about the material provided. If participants feel happy and enjoy the activities and can receive the material well, then the training activities are declared effective or successful (Donald L. Kirkpatrick and James D. Kirkpatrick, 2009;James D. Kirkpatrick, 2016;Phillips, 2003) At present, the above criteria are no longer sufficient to determine the effectiveness of a training activity. This is based on the awareness that good training should not only be enjoyed and received by the participants well but also must be relevant and applicable in their work and contribute to the achievement of organizational goals. This is one of the reasons underlying Kirkpatrick's development of The Four Level Evaluation Model. This model is hierarchical because the evaluation process of training activities must start from level 1 and proceed gradually to the next level. Information obtained from each level will also be the basis for evaluating the next level.
Kirkpatrick's evaluation model is a model that is currently widely used and used as a reference because it is simple, easy to understand and touches almost all aspects of training. However, this model also has drawbacks, including this model does not consider training inputs or participants' backgrounds. Therefore, in practice, this model experiences many adjustments or modifications here and there because it adapts to the needs and characteristics of the organizations that use it. The implementation of Kirkpatrick's evaluation model is expected to provide comprehensive information on the successes and shortcomings of a training activity so that it can be corrected in subsequent training activities.

a. Level 1 -Reaction
As the name implies, level 1 measures how participants react to training activities. The components measured include the level of participant satisfaction with facilities in training, instructors, training schedules, training media, training materials, materials or teaching media, consumption, and so on. Information related to these matters is usually obtained directly from participants through questionnaires or questionnaires distributed shortly after or shortly before the activity ends.
Level 1 is a level that must be evaluated to get information that is useful for the development of training programs. Information obtained from this level is also the basis for evaluating the next level. This is because participant satisfaction will be a motivation and increase the enthusiasm of those involved in participating in learning activities. Conversely, if participants feel dissatisfied with existing conditions, then there is a possibility of motivation and enthusiasm will also be reduced.
If previously this level 1 only evaluates participants' reactions to the implementation of activities, in its current development level 1 also evaluates the involvement and contribution of participants in learning, as well as the relevance of the material to the work of the participants. This is due to the awareness that satisfying or pleasing participants during the course of the activity are not enough. In addition, these two components also further strengthen the role of instructors in training activities because basically, the instructor is as good as any, which is shown by the positive reaction of participants, will be in vain if the ultimate goal of training is not achieved.

b. Level 2 -Learning (Learning)
Level 2 evaluates the extent to which participants learn or understand the material presented, both in terms of knowledge, skills, and attitude. Evaluation of these three components is basically an evaluation of the learning outcomes that take place during the training. Training activities are said to be successful if there are improvements or improvements in the three components, by comparing the results between before and after training. Therefore, the instruments commonly used in this level 2 evaluation are tests, both theoretical tests (pre-test and post-test) and performance tests (practices). The theory test is used to evaluate aspects of knowledge and attitude while practical tests are used to evaluate aspects of skills. Evaluation can also be done by presenting a comparison group that is not included in the training, where this group will be measured and compared to the results of the group that participated in the training.
In addition to evaluating the knowledge, skills, and attitudes of the trainees, the development of learning evaluation is also coupled with a component of confidence and commitment. The confidence component evaluates the extent to which participants believe they can do what they have learned while the commitment component measures the extent to which participants are willing or have a tendency to apply the knowledge and experience gained during training in their work. These two additional components become important because of the expected changes in knowledge, skills and attitude, not only during the training activities but also can be continued and implemented after participants return to their respective workplaces.

c. Level 3 -Behavior
This level measures the extent to which participants apply what they have learned during training after returning to work. In other words, level 3 in Kirkpatrick's model evaluates the change in behavior in question after returning to his workplace. The behavioral changes that are meant here are changes in attitudes or skills for the better. This needs to be done to ensure that the knowledge gained during the training is actually implemented in the workplace and not stored without being used or developed at all.
If the previous two levels evaluate the level of the individual, then evaluation at level 3 involves evaluating at the level of the institution or organization. To measure changes in behavior, the organizer needs to make observations into the work environment of the participants, or through interviews and ask for evaluations from the supervisors and co-workers of the participants. Therefore, getting information related to behavior changes cannot be done shortly after the activity takes place but there must be a time lag to give the opportunity for the participants to return to their work. Some opinions state that the ideal time lag for conducting a level 3 evaluation is the fastest three months after the training ends. In addition, the evaluation should also not only be done once, but several times in a certain period of time to find out whether the behavior change is permanent or not. The components measured at this level are critical behaviors, namely some typical or specific actions that if carried out consistently by the concerned will have an impact on the progress and success of the organization. In its development, level 3 evaluation added two more components, namely required drivers, and on the job learning. Required drivers are processes and systems that apply in the workplace that spur critical behavior such as coaching, and recognition or appreciation of the good performance. On-The-Job learning is also included in the components evaluated because one of the characteristics of modern organizations is the opportunity for employees and leaders to jointly improve performance through the learning process in the workplace.

d. Level 4 -Results
This level measures the extent to which training has an impact on organizational productivity. For organizations that are aiming for profit, the intended effect is an increase in organizational profitability, while for organizations that aim not for profit, the intended effect is the achievement of organizational goals. For example, if the trainee is a teacher, the expected impacts include learning in the classroom to be more enjoyable and students actively participating in learning. If the participant is the leader of the organization or structural official, then the expected impact is, for example, increased teamwork and a more productive and enjoyable work atmosphere.
Same as level 3, this level 4 evaluation is also ideally done through observation, interviews or asking for judgment from others. Level 4 evaluation generally requires a long time because the impact on the organization cannot be seen in a short time. The components evaluated are leading indicators including customer/community satisfaction, employee involvement, quality of work, and other leading indicators according to organizational characteristics.
The implementation of Kirkpatrick's evaluation model in training at the BDK Banjarmasin cannot yet be said to be effective because there are still many obstacles and deficiencies in its application. Level 1 evaluation in the training held by the BDK Banjarmasin has been carried out by collecting assessments from participants on the organizers and widyaiswara. The assessment of the widyaiswara is carried out shortly after the learning ends while the assessment of the committee is carried out before the end of the training activity.
However, based on the authors' observations, there are still some weaknesses and obstacles in its application. The results of observations on several training opportunities showed that the participants' assessment sheets for widyaiswara were not directly distributed after the learning was completed. In fact, there are new ones distributed one day or two days later. This resulted in the tendency of participants to only fill in scores in the form of numbers, while the suggestions and criticism fields were left blank. This can happen because the assessment by the participants is not done immediately after the activity ends so the participants have forgotten the impression they got. This also makes the assessment given not entirely objective.
The results obtained from the participant's assessment also often only become the committee's consumption as material for making the report. In fact, ideally, the results of the assessment, both for the widyaiswara and also the organizers are conveyed to the concerned as an evaluation material other than the leadership as a matter of decision making. This is also influenced by the tendency of the training committee to not directly process the data received so that the data recapitulation process becomes delayed.
The use of printed format in the assessment by participants also makes the process of collecting data recapitulation very time consuming so that there is a tendency for the committee not to input all incoming data but only a few samples. This can actually be dealt with by developing an online charging system which will result in the assessment of participants being directly connected to other instruments so that the results can be seen and known quickly and accurately.
Level 2 evaluation in the training held by the BDK Banjarmasin has been adjusted to the curriculum and syllabus of each training activity. Some are through pre-test and post-test, written examinations, or seminars or presentations. However, in the current curriculum and syllabus, there are still training activities that do not specify the existence of certain learning evaluations. This makes the implementation of level 2 evaluation different from one training to another.
For training that does not specify a specific learning evaluation, pure evaluation or assessment comes from the widyaiswara or the training organizer. This process is also very susceptible to subjectivity because generally the lecturer or committee will give value based on the activity or participation of participants in the learning process only. This will be detrimental to participants who actually have a good understanding but are constrained in delivering or expressing their opinions in the learning forum. Therefore, in addition to using an evaluation form that is in accordance with the existing curriculum and syllabus, the lecturer and committee also need to develop supporting tools or instruments in carrying out this level 2 evaluation in order to get objective results.
In several training activities organized by the BDK Banjarmasin, this critical behavior component was outlined in the form of follow-up plans prepared by participants with direction from widyaiswara. Therefore, in conducting a level 3 evaluation the training should not be separated from the follow-up of participants on the follow-up plans that had been prepared. However, based on the current curriculum and syllabus, not all training activities have established the existence of follow-up plans. This makes level 3 evaluation difficult to do because the evaluator does not have data or material that can be used as a reference in evaluating.
Considering that not all curriculum and syllabus of education and training require the existence of follow-up plans, the BDK Banjarmasin needs to formulate and make one instrument which will later become a reference in conducting this level 3 evaluation in all existing training centers. Instruments can be made in general but it is possible to make them more specific for each type and level of education and training.
The implementation of level 4 evaluation by the BDK Banjarmasin is still very limited considering the many types of training and the characteristics of participants coming from the regions and the different organization which certainly requires a large cost. The evaluation was not entirely arrived at the observation stage or conducted interviews at the workplace but was still limited to asking participants (who are now alumni), superiors, and their colleagues to fill in the instruments already prepared. In fact, ideally, in addition to filling in the instrument, you must also see and know the impact directly on the organization or workplace concerned. Therefore the BDK Banjarmasin needs to develop a more comprehensive instrument so that the evaluation can really measure and get the results that are expected.
The existence of a number of constraints and deficiencies described above shows that the implementation of Kirkpatrick's evaluation model in education at the BDK Banjarmasin is still not effective. This requires a follow-up from the management of the BDK Banjarmasin so that evaluation activities can run well and provide correct results and can be used for the advancement of education in the BDK Banjarmasin in particular and the Ministry of Religion in general.

CONCLUSION
Evaluation is very important in an activity including training. Kirkpatrick's training evaluation model is a training evaluation model developed in the 1950s. This model divides the evaluated components into four levels or levels where each level has its own indicator. The implementation of Kirkpatrick's evaluation model at the BDK Banjarmasin is still ineffective because of the many obstacles on the good field due to a lack of understanding of Kirkpatrick's evaluation model itself, as well as the limited facilities, infrastructure, and available budget.
From the four levels in Kirkpatrick's evaluation model, what is still lacking in the application is level 4, namely an evaluation of the impact of training felt by the organization. As for levels 1 through 3, although it is not yet fully effective, the application has been running quite well. This situations occurs because there are still many employees who do not yet know the four-level evaluation model according to Kirkpatrick and lack of understanding of how the application of the four-level model of evaluation according to Kirkpatrick is good and correct.
As for some suggestions that can be given are to conduct a socialization or workshop on Kirkpatrick's evaluation model from the concept to the preparation of a comprehensive instrument for all employees and instructor, and to increase the utilization of information technology in conducting training evaluations in order to optimize the available budget.