TOSLS Cognitive Instrument to Measure Students' Scientific Literacy Abilities

Objective: This research is for the development of the TOSLS scientific literacy assessment instrument to determine the scientific literacy abilities of undergraduate science education students. Scientific literacy skills need to be known as a form of evaluation in learning. Scientific literacy skills are one of the abilities students must have after graduating from college and facing the work environment. Method: The quantitative method used in this research uses a Rasch model approach from 29 Bachelor of Science Education program students. Indicators of item suitability with Rasch model analysis measure this quality. The questions consist of 9 multiple choice questions with a correct score of 1 and an incorrect score of 0. Data processing uses MS Excel and SPSS 25 software. Results: The results show that all fit items, according to Rasch analysis, can be maintained without revision and used as a question instrument to measure the level of science literacy ability science for students with moderate scientific literacy abilities. Novelty: The research contribution of this question instrument can be used as reference material for evaluating the lecture process in biophysics courses on fermentation topics based on the level of scientific literacy ability.


INTRODUCTION
Science literacy is understanding, interpreting, and using scientific information daily.Scientific solid skills are essential in the increasingly complex and demanding modern world, particularly in science and technology (Rini et al., 2021).The evaluation of Indonesia's science literacy skills conducted by the Programme for International Student Assessment (PISA) shows that Indonesian students' science literacy skills are deficient (Hasan et al., 2018).In 2018, Indonesia ranked 74th out of 79 countries participating in the PISA test (OECD, 2019).One factor affecting students' science literacy skills is more familiarity with standardized science literacy evaluation tests (Suparya et al., 2022).Two main aspects need to be considered to measure the level of science literacy competence: 1) understanding research methods that lead to scientific knowledge; this includes the ability to understand and apply scientific concepts such as hypotheses, variables, and research methods used in science; 2) organizing, analyzing, and interpreting quantitative data and scientific information; measuring skill proficiency aims to diagnose learning difficulties, measure long-term improvements, and gather information to improve performance (Azizah et al., 2020).
Research conducted by Anna in 2023 states that evaluating the Interdisciplinary Science Threshold Experience (InSciTE) results on students' experiences in science using the TOSLS test instrument is more effective in measuring students' science literacy skills in science.Research conducted by Putra et al. (2023) developed a science literacy test instrument that refers to TOSLS specifically for physics education students.As a form of literacy science ability test, in this study, the researchers wanted to know the quality of the instrument designed to measure or determine students' scientific abilities through the biophysics course on fermentation.There is a relationship between the topic of fermentation and science literacy among students, as fermentation is a scientific process that requires an understanding of scientific concepts such as microbiology and biochemistry results; it can be concluded that an instrument test with a topic on fermentation can be used comprehensively for students majoring in science, biology, and chemistry (Rezeqi & Gultom, 2023).Based on the above explanation, a valid cognitive test instrument was developed for students that can be used to assess their science literacy skills.Having an assessment of these skills can help improve students' science literacy skills as they become familiar with science literacy-based questions and become a basis for further education/teaching evaluation.

RESEARCH METHOD
This research used the ADDIE development model.The lecture achievement was assessed in the analysis phase, and the material topic was determined.In the design phase, a test instrument grid was designed.In the development phase, the questionnaire was developed, consisting of nine multiple-choice questions, with a correct score of 1 and an incorrect score of 0. The subjects of this research were 29 students of the Bachelor of Science Education program at Surabaya State University.The research method stages are described in Figure 1.This research focuses on analyzing evaluation instruments using the Rasch model.Therefore, the data obtained will be binary.The results were analyzed using the extended Rasch model in Ms.Excel and SPSS.From the resulting output, several fitting parameters were determined using the Rasch model.The MNSQ outfit value is between 0.5 and 1.5.

Results
The output results from SPSS show that all items fit with MNSQ outfit values between 0.5 and 1.5.Each item correlates with the research subject, showing that the questions created have shown that they are in all categories, namely easy, medium, and challenging.Question item number 5 is a question item in the easy category because the graph shows that the peak point of the curve is around -4.Meanwhile, for item number 3 on the x-axis, the score was -5 to -1; no students answered correctly, and the peak point for item number 3 was between 4-6, meaning that students with high scientific literacy skills can answer this question.The difficulty level of the questions is shown in Figure 2.

Discussion
Scientific literacy is an individual's ability to use scientific knowledge, identify questions, and draw conclusions based on valid evidence.Scientific literacy competency provides a perspective on societal problems in the current digital era, which is increasingly dependent on the use of technology in the development of science (Suwandi & Supriyanti, 2021).One way to measure students' scientific literacy abilities is to use scientific literacy-based assessments (Hendri & Hasriani, 2019;Saraswati et al., 2021).Cognitive assessment measures and evaluates a person's cognitive skills, such as students' scientific skills (Rahmayanti et al., 2021).A cognitive assessment aims to determine how well a person understands scientific concepts, can apply scientific knowledge to real-world situations, and can think critically and analytically (Lubis et al., 2021).Test of Scientific Literacy Skills (TOSLS) is an instrument or test designed to measure literacy skills using scientific literacy indicators developed by Gromally et al. (2012).Scientific literacy test instruments should be developed using appropriate indicators, including scientific phenomena., analysis of valid scientific sources and graphic data.The questions are prepared using nine multiple-choice questions.The objective test for scientific literacy questions is usually multiple-choice choices equipped with a scoring rubric (Istyadji, 2023;Masfuah et al., 2021;Nurhayati et al., 2023;Polat, 2020;Yamtinah et al., 2022).If the answer is correct, the score is 1; if the answer is incorrect, the score is 0 (Arikunto, 2017).The form of a scientific literacy test instrument can be a multiple choice objective test/description/attitude and skills test (Fuadah, 2017).
Based on the data obtained, the 9 test items developed can measure students' scientific literacy abilities, with the range of fit results for all items being 0.5 to 1.5.Test instruments that meet the MNSQ outfit value of 0.5-1.5 can be maintained without modification (Saragih et al., 2023).Student scientific literacy instruments are said to be usable if all aspects of scientific literacy indicators are listed on the instrument in the fit category according to analysis using the Rasch method (Krisanda & Harjito, 2021).In the Rasch model, test participants with high abilities are likelier to answer correctly than other students.On the other hand, students who have scientific literacy abilities will answer more complex questions (Rusilowati, 2018).The difficulty level of the items needs to be considered and adjusted to the student's initial abilities.The level of difficulty is a measure of the ease of the question, where questions in the easy category mean that it is easy for the question to be answered correctly, while questions in the difficult category mean that it is difficult for students to answer correctly (Adawiyah & Wisudawati, 2017).
Question item Question number 3 is a question with a high difficulty level.This shows that those who can answer question number 3 are students who have high scientific literacy abilities.The graph shows the point at a value >3.The scientific literacy indicator in item 3 evaluates the use and misuse of valid scientific information.This shows that students' ability to carry out evaluations could be higher.The evaluation stage will start with analytical skills and does not depend on memorization alone but rather on analytical skills in understanding a problem so that it can provide the correct conclusions (Irwan et al., 2020).A factor that can influence this ability is that students need to be used to working on questions in the form of analysis and evaluation questions.Currently, test instruments used in education usually only use aspects of scientific knowledge from the perspective of content and understanding for analysis and evaluation aspects as a supporting role in the questions (Chasanah et al., 2022).Students are also less able to identify valid opinions and scientific sources closely related to aspects of scientific knowledge they understand, so they could be more helpful in answering questions correctly.Cognitive theory explains that, in this case, students are using their initial knowledge to process new information using the initial knowledge they already have (Novitasari, 2018).Of course, good scientific knowledge is expected to increase understanding of scientific concepts.
Question item 5 is a question level in the easy category with question indicators, namely presenting research data related to calculations.Students conclude the calculation data by selecting the correct data graph in the choice of questions.Question number 5 can be used for students with low scientific literacy ability categories.In contrast, question item number 5 is not suitable for measuring high levels of scientific literacy ability because it will feel straightforward to answer for them.Students can quickly answer this question because they are used to interpreting data from tabular form to graphs (Marlina, 2019).Developing a cognitive test instrument for fermentation material in biophysics lectures is effective for students with moderate scientific literacy skills (Anna et al., 2023).A cognitive instrument is a tool that can be used to measure the achievement of a practical goal and obtain results as an evaluation process.In another research by Suryani et al. 2015, a test instrument developed that included scientific literacy indicators could measure the level of students' scientific literacy abilities.Scientific literacy can be applied to courses where learning outcomes require analysis and evaluation skills (Pujiastutik, 2018).Farwati et al. (2018), several things influence students' scientific literacy levels, including internal and external factors.Studying regularly, getting used to using valid sources, and answering or taking tests with scientific literacy-based instruments can help improve scientific literacy skills (Sumanik et al., 2021).Other factors that can influence are external factors, including the availability of student activity sheets and teaching materials based on scientific literacy (Ramlawati et al., 2022).The evaluation results of the level of scientific literacy abilities can be improved through a learning improvement process that does not emphasize only mastery of concepts (Rusydiana et al., 2023;Sholahuddin et al., 2021).To overcome students' low scientific literacy skills, teaching materials, student activity sheets, and assessment instruments that refer to scientific literacy indicators are used (Sumanik et al., 2021).Therefore, it is necessary to develop question instruments using scientific literacy to evaluate student abilities and learning in educational units.

CONCLUSION Fundamental Finding:
The TOSLS cognitive question instrument can be used to measure the scientific literacy abilities of students with moderate scientific literacy abilities in biophysics on fermentation.All question items are by the Rasch analysis model so that all question items can be used without revision.Cognitive assessment in the form of 9 multiple-choice questions can be used to assess scientific literacy.Implication: The existence of this instrument can be used as an instrument for evaluating the learning process and improving students' scientific literacy abilities.Limitation: The question instrument developed can only be used on fermented materials.Future research: This needs to be developed with several questions in other forms than multiple choice and using other subject topics to start the skills and learning process regularly.

Figure 2 .
Figure 2. Correlation of difficulty levels of question items.