Development of Multi-Representation Test As A Solution to Train High- Order Thinking Skills High School Students in Newton’s Law

Sections Info ABSTRACT Article history: Submitted: December 23, 2020 Final Revised: January 14, 2021 Accepted: January 14, 2021 Published Online: January 31, 2021 This research aims to develop a multi-representation based test instrument that can be used to measure students' higher-order thinking skills, especially in Newton's law material. development procedures used the Plomp development model, the stages were design, construction/ realization, test, evaluation, revision, and implementation. The subjects in this study were 36 students of class X at one of High School in Surabaya. At the implementation stage, tests were given to students and analysed using Rasch analysis with help of Winstep software. The multirepresentation test instrument in question was a question in the form of an essay with a representation model consisted of visual, verbal, and mathematical representations adapted to the cognitive domain of Bloom's taxonomy of higher-order thinking. Data collection techniques were validation of instruments and tests. The results of this study were 9 items of valid test instruments based on logical validity and empirical validity and a reliable instrument based on calculations using the Alpha Cronbach equation. Based on the results of this research can be concluded that multi-representation test can be train high order thinking skills students. Study with multi-representation test is expected to be able to make students are easier to develop high order thinking skill, in this research students can be categorized as having sufficient high-order thinking skills.


INTRODUCTION
Physics has an important role in improving the quality of human resources, as a development of natural science and technology (Halim, 2018). Because of the importance of physics to global challenges and technological advances, so the learning process is required to be able to produce human resources who have high intellectual, curiosity, self-confidence, and have the skills to develop knowledge as provisions in achieving educational goals. The 2013 curriculum was designed with various improvements from the previous curriculum that are adapted to education at the international level, including the content standards and assessment standards. The content standard is expanded with material that is relevant to the needs of students as a support to be able to think critically and analytically, the assessment standards used are adapted to an assessment that is adjusted to international standards.
The assessment is expected to help students improve their higher-order thinking skills. The thinking ability must continue to develop to form individuals who are successful in facing all challenges. By having a high thinking ability, every problem encountered can be resolved properly, students are able to survive in any condition and gain success in life (Wasis et al., 2020). An important goal in the world of education today is to direct the development of high-level thinking skills of students because it is the ability to think highly that can make someone exist to confront the challenges of the period so that education in Indonesia has prepared human resources (Zohar & Cohen, 2016).
Higher-Order Thinking is high-order thinking that thinks logically, critically, reflective, metacognitive, and creatively (Anderson & Krathwohl, 2001). Higher-order thinking skills are a person's ability to be able to connect, manipulate, and transform their knowledge and experience to think critically and creatively in making decisions and solving problems in new situations (Winarni, 2019). Higher-order thinking skills are abilities that require students to apply and manipulate new information or knowledge they get to achieve possible answers in new situations. One of the international studies organized by the OECD (Organization for Economic Cooperation and Development) is PISA (Program for International Student Assessment). Based on the results of the PISA study, it can be seen that the achievement of Indonesian students' reading literacy, mathematical literacy, and scientific literacy is still low. The results of the achievement of Indonesian students' scientific literacy on the PISA study test in 2012 were ranked 64 out of 65 countries, in 2015 Indonesia's PISA test results were ranked 69 out of 76 countries, while in 2018 Indonesia's PISA test results were ranked 71 out of 79 countries. This indicates that Indonesian students still have low average ability. So it is necessary to evaluate and reform the education system in Indonesia. Improving the assessment system is one of the objectives that the Ministry of Education and Culture is currently reviewing. The assessment is designed so that it can be reported in a form that is useful for the improvement and formulation of education policies. Therefore, to catch up with Indonesia's education from other countries, it is necessary to present questions based on high order thinking so that students can hone their thinking skills. Studying of physics can be presented with various representations, they are formulas, calculations, graphs, and conceptual explanations that are presented simultaneously. In mastering physics, in addition to understanding the concept, you must also have the correct mathematical, logical, and intuitive skills (Setyani et al., 2016). Studying physics will be easy to understand if it is presented in a different representation format to support students' higher-order thinking skills. Therefore, a stimulus is needed to students by giving test questions in the form of multiple representations, because each student has different abilities, giving a multirepresentation test will help students improve their thinking skills based on their representational abilities. Research on multi-representation can help students in multirepresentation problem solving by identifying, planning, executing, and evaluating (Siswanto et al., 2020).
According to Sutopo & Waldrip (2014) in learning physics using a multirepresentation approach can improve students' representational abilities and improve student reasoning through the use of representations. Representation is important for students' physics learning because the content presented is the same can be obtained in different ways (Franco et al., 2012). Students will study physics more effectively and efficiently when using multiple representations (Huda et al., 2016). Multiple representations can foster an understanding of physics concepts. According to Suhandi & Wibowo (2012), the multi-representation approach is effective enough to foster students' understanding of physics concepts. Three main functions of representation: IJORER: https://journal.ia-education.com/index.php/ijorer first, representations can complement each other; second, can explain interpretation; and third, the combination of several representations can help students organize understanding in the topic being studied. Studying physics using multirepresentation will help students in understanding the concepts of physics so they can resolve learning difficulties experienced by students (Sinaga & Suhandi, 2014).
Newton's law material is a matter of physics which is the basic concept of classical mechanics. In the basic concept of force, students do not understand and master the concept of Newton's law, so the next material students will experience difficulties (Halim et al., 2014). There are still many students who still do not understand the basic concepts of style, there are misconceptions, and conceptual understanding (Alias et al., 2016;Ergin, 2016). Therefore, it is necessary to continue to develop related to Newton's law. Seeing the importance of students in higher-order thinking skills, multirepresentation test questions are needed as a solution in fostering students' higherorder thinking skills.

RESEARCH METHOD General Background
This research was development research to develop a test instrument multirepresentation question. Data analysis used descriptive quantitative. The test instrument development procedure used the Plomp development model. The research phase consisted of 5 phases, were: 1) investigation phase, 2) design phase, 3) realization/construction, 4) test, evaluation, and revision, and 5) implementation phase (Plomp et al., 2010) in Figure 1.

Participants
This research was conducted at one of the high schools in Surabaya. Implementation was in the academic year 2019/2020. The population in this research were all students of senior high school 17 Surabaya. Sampling in this research was determined by cluster random sampling, the trial subjects were 36 students. The quantitative data collection in this research data obtained through a multi-representation test.

Instrument and Procedures
This type of research is development research to development product. Development research used to products and get the effectiveness of the products (Astutik & Prahani, 2018). The instruments and procedures used in this study were 1) The validation sheet as an assessment given by the validator by giving a check (√), 2) empirical validity analysis with Rasch modeling analysis used Winstep software, 3) reliability analysis used the alpha Cronbach equation, 4) study test used multi-representation questions.

Data Analysis
After the test instrument has been developed, the next step is to find out its validity and reliability. Validity consists of two, they are logical and empirical validity (Riduwan, 2015). Logical validity is based on the assessment of physics expert lecturers, by validating the test instruments on physics expert lecturers at the State University of Surabaya which aims to give an assessment of each item on the multi-representation test based on the material, construct, and language aspects. The validation scoring system uses a scale of 1-4, namely (1: not good, 2: not good, 3: good, 4: very good). The results of the multi-representation instrument test validation were then analyzed descriptively quantitatively, were calculating the average score of the scores given by the validator. From the results of the validator's average score, then determine the criteria presented in Tabel 1. Can't use and still requires consultation (Ratumanan & Laurens, 2011) The reliability of the test instrument validation results based on the inter-observer agreement obtained from the precentage of agreement (R) analysis and reliable if the Rvalue is above 75% (Borich, 1994).
For information: = The reliability coefficient of the validation results = The highest score of the 2 validators = The lowest score of the 2 validators After the multi-representation test instrument was known to be valid or not, the next step was to test on 36 students to test its empirical validity and reliability. The empirical validity was assessed used Rasch analysis with help Winstep software. With the output generated from the Winstep software, we can find out the overall information about the questions that were valid based on the specified criteria. Results of the Instrument test were valid if they had the following requirements in Table 2.

RESULTS AND DISCUSSION
The development of a multi-representation test instrument can be used as a reference or an image description of questions that can measure high order thinking skills of students so that it is expected to improve the quality of student learning in higher-order thinking. Based on the statement stated by Wasis et al. (2020) that assessment will not improve learning outcomes, but a well-designed assessment will be able to determine/guide quality learning so that learning outcomes are of higher quality. The results of this study are expected to provide educators with an overview of the types of questions that can help students improve their higher-order thinking skills, especially questions related to multi-representation. This research went through several stages, the first step the researcher did was developing a multi-representation test instrument. Development design using the Plomp development model. There are 5 phases of development, namely investigation, design, realization, testing, and implementation (Plomp et al., 2010). Based on these stages, 3 packages of multi-representation questions were produced to measure higher-order thinking skills. Each test question package consists of 3 multi-representative essay questions with measured high-order thinking skills including the ability to analysis (C4), evaluate (C5), and create (C6). The multirepresentation problem in question is a question in the form of visual, verbal, and mathematical representations. Indicators and forms of multi-representation test representations can be seen in the Table 3. The developed test questions were in the form of essay questions, the purpose of developed essay questions by the researcher was to determine the level of analysis and students' abilities which were then used as an analysis of students' high-order thinking skills based on students' results and answers to multi-representation test questions. This is in accordance with the opinion of Suwarto (2013) which states that the description questions can assess the level of understanding, give students freedom in developing answers based on students 'own ideas, and can show students' ability to organize and create solutions. In the design stage of developing a multi-representation test based on Newton's law material indicators that accord Curriculum 2013, multi-representation questions are generated. The following is an example of a developed test multirepresentation in Table 4. The assessment is based on the scoring rubric that has been developed. This is an example of a scoring rubric in the example problem above, see Table 5. A good test instrument is not only valid but also reliable. The feasibility of a multirepresentation test instrument is based on the validity and reliability of the test instrument. The test instrument can be trusted for research, so the test instrument requires validation and reliability analysis (Samsudin, 2020). The validity of the instrument tests can be divided into two, they are logical validity and empirical validity. The results of logical validity can see in Table 6. The average result of the assessment of the multi-representation test instrument validation is categorized as a valid product with an average score of more than 2.5 on all multi-representation test items. The product has been declared valid with a slight revision if the average score obtained from the validator is more than 2.5 and less than 3.25, it can see on Table 1 (Ratumanan & Laurens, 2011). So it can be stated that the developed multi-representation test questions are valid and can be used with several revisions ( Table 6). The results of the test of suitability (percentage of agreement) of the two validators on all multi-presentation test items were more than 75%. The reliability of the test instrument validation results is based on the inter-observer agreement obtained from the precentage of agreement (R) analysis and is said to be reliable if the R-value is more than 75% (Borich, 1994). The test instrument for the empirical validity test used Winstep software to determine the level of suitability of the items (item fit/valid items). The test results are valid if they have the requirements according to Table 2. The items are valid if the results of the analysis have two conditions which are valid criteria (Boone et al., 2014;Sumintono et al., 2015). The result of empirical validity used Winstep software can see in Table 7. It is known that the items had the criteria for the minimum requirements specified. Based on Table 7, the items that meet the requirements are marked with numbers in bold. Therefore, the developed multi-representation test questions are valid and suitable for use as a test of higher-order thinking skills. A valid multi-representation test can then be used as a student trial to measure higher-order thinking skills. A multipresentation test that contains valid high-order thinking questions encourages students to think deeply about the material being studied (Barnet & Francis, 2012). Reliability was tested using the Cronbach Alpa equation (r11) analysis using Winstep software. Person RAW Score-to-measure correlation = .96 Cronbach alpha (r11) Person Raw Score "Test" Reliability = .68 SEM = 1.42 Person RAW Score-to-measure correlation = 1.00 Cronbach alpha (r11) Person Raw Score "Test" Reliability = .80 SEM = 1.28 Person RAW Score-to-measure correlation = 1.00 Cronbach alpha (r11) Person Raw Score "Test" Reliability = . 85 SEM = .99 Reliability is the scope of an instrument consistently and can be integrated with the truth convincing measurement (Tiruneh et al., 2017). The Cronbach alpha (r11) value or the reliability of question package 1 was obtained at 0.68, which means that the multirepresentation test instrument in package 1 that had been developed was classified as moderate. The Cronbach alpha (r11) value or the reliability of question package 2 was obtained at 0.80, which means that the multi-representation test in package 2 that had been developed was high. The Cronbach alpha (r11) value or the reliability of the question pack was obtained at 0.85, which means that the multi-representation test instrument in package 3 that has been developed is high. Judging from the logical validity, empirical validity, and reliability of the questions, the three packages on the multi-representation test instrument as a whole are feasible and can be used as a measure of students' higher-order thinking abilities. These results have met the criteria for a good instrument according to Sudaryono (2017) which states that a good instrument is a valid and reliable instrument.
The following is an illustration of higher-order thinking skills from the test results using a multi-representation test. The diagram of students' higher-order thinking skills based on the ability criteria and indicators of higher-order thinking skills is presented in Figure 1.  Figure 2 gives information that the average high-order thinking ability as a whole can be seen that the ability to analysis students gets an average score of 47, the ability to evaluate gets an average score of 62, and the ability of students to be create got an average score of 54. So that the average high-level thinking ability of high school students in Surabaya is 54. Students who have high-order thinking skills with very high categories, these students can master multi-representations that are presented well. Whether on the ability to represent visual, verbal, or mathematical. These students are able to understand concepts and solve problems. Theasy et al., (2018) states that students who are categorized as having high-order thinking skills are able to solve problems well are able to understand the meaning of words and relate them to physics concepts. Multiple representations can be used to differentiate students' conceptual understanding. Students who can state the concept well will have no difficulty directing their understanding in various forms of representation (Sutopo & Waldrip, 2014). Sirait (2016) found that many students get good results in solving problems with a process that is precede by visualization using sketches or diagrams (physical representations). The resolving of visual representations is considered to make it easier for students to solve problems in questions. Sutarto et al. (2018) provides a verbal representation (picture) of the students that have a significant By developing multi-representation questions, it will make it easier for students to practice higher-order thinking skills. Because higher order thinking is a thinking activity that not only states facts, but is more important than facts. So the primary thing to do is understand facts, connect facts, categorize facts, manipulate facts, and use these facts in new situations to get new solutions to a problem (Ramos et al., 2013). The use of various representations helps students in solving physics problems (Pratama et al., 2018;Jannah et al., 2019). Students' ability to build and understand multiple representations has an important aspect for building knowledge (Taqwa et al., 2020). (Sutopo & Waldrip, 2014) the ability to use representations is measured as a significant presence for physics learning.

CONCLUSIONS
The test instrument developed on the subject of Newton's law consists of 9 description items and was divided into 3 question packages that had the eligibility criteria as a high-level thinking test instrument which was good in terms of logical and empirical validity. Valid was reflected in the results of the expert validator's assessment, all validators state the questions based on material, construct, and language. Meanwhile, empirically based on the results of the student score analysis and all questions were categorized as valid. Based on the analysis using Winstep software, the question package can be declared reliable. Multi-representation tests can be used as a measuring tool for measuring students' high-order thinking skills. Each individual has different abilities so that multi-representation questions will make it easier for students to increase their interest in learning, especially in higher-order thinking skills. High-order thinking skills of students of Senior high school 17 Surabaya are classified as sufficient with an overall average score of 54. The order of students' high-order thinking skills from highest to low is the ability to evaluate, create, and analyze. Suggestions to future researchers are expected to use more representations used on different subject matter topics to practice higher-order thinking skills. In physics learning, the researcher is expected to provide learning with a multi-representation approach as an understanding of students' physics concepts.