The Effectiveness of The Six Tier Diagnostic Test (STDT) Instrument in Viewed from Empirical Validity to Identify Student's Misconceptions in Chemical Equilibrium Materials

ABSTRACT


INTRODUCTION
Teaching chemistry as a branch of science to students should refer to a philosophical foundation in the development of science, which includes three scientific components: scientific product, scientific method, and scientific attitude.The chemical content taught is a scientific product that includes facts, concepts, principles, laws, and theories.In addition to paying attention to scientific products, planning and implementing chemistry lessons must include the scientific method and a scientific attitude (Istikhomah & Wachid, 2021).Students are stated to be able to master the concept comprehensively if they can answer three scientific questions related to the concept being studied.The three scientific questions are ontology, epistemology, and axiology.Ontology, epistemology, and axiology questions are marked with the keywords what, how, and what are the benefits, respectively.
The concept is defined as a generalization of facts with the same characteristics.The concept is abstract because it does not have its existence other than in the individual's mental representation.The results of generalizing (abstraction) facts that have been accommodated in one's schema are called conceptions (Wahyuni, 2018).Conception is a person's perspective on a concept.Suppose the results of an individual's abstraction of a concept are not by the scientific understanding or understanding received by scientists in the field concerned.In that case, this individual is declared to have a misconception (Yuliati, 2017).If learning chemistry concepts facilitates the process of constructing knowledge and organizing information into students' cognitive structures in a comprehensive manner, then the possibility of misconceptions can be minimized (Rosita et al., 2020;Şen, & Yilmaz, 2017).
Students have a comprehensive conception of chemistry; correctly viewed from the dimensions of ontology, epistemology, and axiology, it can be determined as an achievement of chemistry learning.It is necessary to develop instruments to diagnose conceptions or misconceptions about the chemical concepts that have been learned.Through this instrument, it will be determined whether a person has a misconception or not on certain chemical concepts.To determine whether someone has a misconception or not, a representative instrument is needed.Representation of the dimensions of ontology, epistemology, and axiology in the package of questions on mastering chemistry concepts becomes urgent when one wants to take a comprehensive picture of one's conception.Diagnostic tests determine weaknesses in students' understanding of a concept (Anam, 2019;Prodjosantoso, 2019).An excellent diagnostic test can provide an accurate picture of the misconceptions experienced by students based on the error information they make (Bayuni, 2018;Maharani, 2019;Qonita & Ermawati, 2020).Good diagnostic questions not only show that students do not understand certain parts of the material but can also show how students think in answering the questions given, even though the answers are not correct (Setiawan, 2020;Sari, 2020;Timothy, 2023).Suppose this statement is linked to three scientific questions.In that case, an excellent diagnostic question will not only show that students do not understand the ontology dimension but also how students think about how concepts are constructed (epistemology) and the benefits or values of a concept (axiology).Therefore, it is necessary to develop instruments to measure students' conceptual understanding of the three scientific questions.
The development of research on diagnostic tests has reached the five-tier diagnostic test.However, the diagnostic test instrument has yet to reach the level of understanding of the use value (axiological aspect) of the concept.In analyzing the depth of concepts measured by the instrument, the instrument still only measures concepts in the ontology aspect.Based on this, it is necessary to improve the development of diagnostic test instruments into six-tier diagnostic tests, which can measure students' understanding of concepts comprehensively.Further development of existing diagnostic test instruments is needed.The diagnostic test instrument that was developed next was the six-tier diagnostic test, referred to as the STDT instrument.The STDT instrument developed is capable of analyzing students' conceptual understanding comprehensively from aspects of ontology, epistemology, to axiology.
The STDT instrument format is multiple-choice questions with five answer choices in the first, third, and fifth tiers.The first tier is in the form of questions on the ontology aspect, the third tier is a question on the epistemological aspect, and the fifth tier is a question on the axiological aspect.In addition to the multiple choice questions, the STDT instrument also contains confidence questions for the level of student answers located in the second, fourth, and sixth tiers.Confidence level questions are presented as a certainty of response index (CRI), which contains students' confidence levels on a scale of 0 to 5 (Borovicza & Szarka, 2023).
Several studies have shown that chemical equilibrium is an essential topic in the study of chemistry learning but is difficult to understand (Mai, 2021).Usu (2019), Siswaningsih (2019), andHarza (2021) argued that students often experience misconceptions about chemical equilibrium, especially in the dynamic equilibrium subtopic.Siswaningsih (2017), Jusniar (2019), Yamtinah (2019), Khairunnisa & Prodjosantoso (2020), Andriani, et al., (2020), and Jusniar (2020) suggested that apart from dynamic equilibrium, misconceptions about chemical equilibrium also occur in the sub-topics of equilibrium shifts and equilibrium constants.The research results regarding misconceptions about other chemical equilibriums developed by Omilani and Elebute (2020) suggest that misconceptions about chemical equilibrium material also occur in the sub-topic of factors that influence shifts in equilibrium.Based on several research results, it was concluded that there are still things that need to be clarified about chemical equilibrium material.Identification of misconceptions must use instruments that can measure students' understanding of concepts in a representative way.In this article, we will discuss the empirical validity of the STDT instruments that have been developed.

RESEARCH METHOD
This type of research is Research and Development (R&D).Development research is a method used to produce specific products and to test the effectiveness of these products (Sugiono, 2017).The development of the SMA chemistry STDT instrument refers to the ADDIE development model.The ADDIE model consists of five stages, namely analysis and evaluation, design and evaluation, development and evaluation, implementation and evaluation, and evaluation (Branch, 2009).(Branch, 2009).
The STDT instrument developed was implemented on a sample of 86 students from a total population of 108 students at Hang Tuah 5 Sidoarjo Senior High School.The results of instrument implementation are used to analyze the reliability and effectiveness of the instrument, which refers to empirical validity.The reliability of the instrument was analyzed using the Cronbach's Alpha method.The effectiveness of the instrument is tested by correlating the distribution pattern of the level of misconceptions with the design level of ease or level of difficulty of the questions.The distribution pattern of student misconception levels was obtained from the results of data analysis after students were tested using the STDT instrument.After carrying out the instrument test, data was obtained on the percentage of students who experienced misconceptions.Data on the percentage of students who experience misconceptions is then categorized into three intervals, high, medium, and low, for each question item.The level of ease and difficulty of the questions is obtained by analyzing the percentage of students' correct answers, which are then categorized into easy, medium, and challenging questions.Data on the level of misconceptions and difficulty of questions were then tested using chi-square statistical analysis.

Type of Conception
The application of the STDT instrument measures the level of students' conceptions.Determination of the category of students' conception level is determined based on various possible student answers at all levels of questions by the rubric of the instrument.Based on the analysis of students' answers, categorizing the level of students' conceptions was carried out.Table 1 shows the students' proposed conception levels in the six-tier diagnostic test format.Respondents gave all correct answers on the ontology, epistemology, and axiology aspects of the concept.Overall, confidence levels were high in answering all three aspects of the question.

LC (Lack of Confidence)
Respondents gave all correct answers on the ontology, epistemology, and axiology aspects of the concept.There is at least one answer with a low confidence level in one aspect of the question.LK (Lack of Knowledge) Respondents gave at least one correct answer to questions regarding aspects of ontology, epistemology, and axiology of concepts.There is at least one answer with a low confidence level in one aspect of the question.

MSC (Misconception)
Respondents gave all wrong answers on the ontology, epistemology, and axiology aspects of the concept.Overall, confidence levels were high in answering all three aspects of the question.

HNC (Have No Conception)
Respondents gave all wrong answers on the ontology, epistemology, and axiology aspects of the concept.All confidence levels are low in answering all three aspects of the question.
Conception level data will then focus on data on the percentage of students who experience misconceptions about each item.This is done to determine the instrument's effectiveness through the empirical validity of the results of students' answers.Data on the percentage of students who experience misconceptions will be tested using chisquared with the difficulty level of the items.Before that, the data on the percentage of students who experienced misconceptions were categorized into small, medium, and large numbers with the same data range intervals.Based on Table 2, there are ten questions with a low percentage of students having misconceptions, four with a moderate percentage of students experiencing misconceptions, and 1 question with a problematic percentage of students experiencing misconceptions.

Relationship Students' Misconceptions and Difficulty Level of Question Items
The application of the STDT instrument is also used to analyze the level of difficulty of the instrument questions.The difficulty level of a question item is the proportion of students who answer correctly.The more significant the p-value (difficulty level index) of a question, the easier the problem is.Conversely, the smaller the p-value (difficulty level index), the more complex the item is.The difficulty index data for each item on the STDT instrument is presented in Table 3.The results of testing the data on the percentage of students who experience misconceptions about the difficulty of the 15 items on the STDT instrument are presented in Tables 4 and 5. Based on the data in Table 4, the number of questions at the medium difficulty level is 13 questions, and at the complex level is two questions.The percentage of students who have a little misconception at the difficulty level of easy questions does not exist; at the medium difficulty level, there are ten questions, and there are no complicated questions.The percentage of students with moderate misconceptions at the difficulty level of easy questions did not exist; at the medium difficulty level, there were three questions, and at the difficulty level for complex questions, 1.The percentage of students with many misconceptions at the easy problem level did not exist, medium difficulty level did not exist, and difficult item difficulty level 1.Based on the data from the chi-square test results above, the Asymp value is obtained.Sig.(2-sided) 0.014.Referring to the basis for decision-making previously described, the Asymp.Sig.(2-sided) 0.014 <0.05, it can be concluded that H0 is rejected and Ha is accepted.This shows a relationship between the difficulty of the items and the percentage of students who experience misconceptions.These results indicate the instrument's effectiveness in terms of the empirical validity of the results of the application of the STDT instrument.
To support the instrument effectiveness data, which refers to empirical validity data, the instrument reliability data is also analyzed.Reliability testing can be used to determine the consistency of a measuring instrument and whether the measuring instrument remains consistent if the measurement is repeated.A measuring instrument is reliable if it produces the same results, even if measurements are taken many times.The results of the reliability test using Cronbach's Alpha method produced an instrument reliability coefficient (r) of 0.815.The rtable value used for N=86 or df=N-2 is df=84 with a significance level of 5% or 0.05, namely 0.212.With these results, the test instrument that has been developed is reliable because r count > r table.

Discussion
After the implementation of the instrument test, data on the level of students' conceptions was obtained, which was then narrowed down to focus on the percentage of students who had misconceptions.Data on the percentage of students who experience misconceptions are then categorized into three intervals, namely high, medium, and low, for each item.The level of difficulty of the questions is obtained from an analysis of the percentage of correctness of student answers, which are then also categorized into easy, medium, and challenging questions.Data on the level of misconceptions and difficulty of the questions were then tested using chi-square statistical analysis.
The STDT instrument was developed to measure the level of students' conceptions, especially to detect students' misconceptions.The results of the application of the instrument showed that there were students who needed clarification about each item.The results of the highest misconceptions were found in item number 13, with the number of students experiencing misconceptions at 26.97%.This percentage value will be the highest reference for students who experience misconceptions.Three ranges of values will be made from 0 to 26.97%, which will then be given the few, medium, and many categories.Of the 15 questions on the STDT instrument that were developed, ten questions were in the category of the number of students who experienced slight misconceptions, four questions were in the category of the number of students who experienced moderate misconceptions, and 1 item in the category of the number of students who experienced a large number of misconceptions.
Other data obtained is data on the difficulty of the items, which are categorized as easy, medium, and challenging questions.The categorization of questions in the STDT instrument refers to the analysis of empirical data on the percentage of correctness of student answers (Haeruddin et al., 2020;Karunia & Ridlo, 2022;Wuryandani & Herwin, 2021).Based on the analysis of student answers, there were 13 questions in the moderate category and two in the problematic category.There were no questions in the easy category; 86.67% of the questions were in the medium category, and 13.33% were in the difficult category.The composition of the item difficulty level in a package of questions ideally consists of 25.00% easy questions, 50.00% medium questions, and 25% difficult questions (Rahmi et al., 2022).Initially, the preparation of the test items on the STDT instrument only paid attention to the focus on the indicators of instrument development; namely, the items contained aspects of ontology, epistemology, and axiology comprehensively.The composition of the difficulty level of the test items had yet to be considered at the beginning of the preparation of the items on the STDT instrument.This needs to be a concern in preparing the next question package.
Data on the number of students who experienced misconceptions and data on the level of difficulty of each item were then correlated with the chi-square statistical test.The chi-square test helps test the relationship or influence of two variables and measuring the strength of the relationship between one variable and another (Djarwanto, 1985).In this study, the strength of the relationship was tested for two variables, namely the level of difficulty of the items and the number of students who had misconceptions about each item.The chi-square test that has been done shows that H0 is rejected and Ha is accepted.This can be interpreted if there is a relationship between the difficulty of the items and misconceptions.Based on the results of these statistical tests, it can be concluded that the effectiveness of the instrument has been fulfilled (Artyushkova, 2020;Hoekstra et al., 2019;Lindner et al., 2020;Van Smeden et al., 2020).The instrument can identify misconceptions experienced by students based on the distribution of difficulty levels of the items developed.The discussion regarding the validity, practicality, and effectiveness of the above instruments refers to a conclusion that the developed STDT instrument meets the valid, practical, and effective criteria.The STDT instrument is feasible to measure students' conceptions, especially in detecting misconceptions.Of course, the level of conceptions, especially misconceptions that are detected, is comprehensively measurable in terms of three scientific questions, namely ontology, epistemology, and axiology.

CONCLUSION Fundamental Finding:
A valid six-tier diagnostic test instrument was produced to detect students' misconceptions about chemical equilibrium material.Implication: STDT can be used by teachers to detect students' conceptions of chemical equilibrium material and become a reference for other researchers to develop similar research.Limitation: The STDT that has been developed integrates three scientific questions, where the three scientific questions are benchmarks of comprehensive understanding.Future Research: STDT can be developed to diagnose students' types of conceptions in other materials.

Table 1 .
Description of student's conception level in the six-tier diagnostic test instrument.

Table 2 .
Number of students' misconceptions on each question item.

Table 3 .
Question item difficulty index.

Table 4 .
Crossing data on the percentage of misconceptions with problem difficulties.