Data Collection Procedures

• Validity refers to whether we are truly measuring the concept of interest in our study.
• An instrument refers to the questionnaire or data gathering tool to be constructed, validated, and administered.
• An instrument cannot be valid if it is not reliable.
• False: In measuring the speaking proficiency of the students, administering multiple choice type of tests produces a valid result.
• When you are performing test-retest reliability, you will be given the same instrument twice to the same group of people.
• 3 ways to have research instruments:

  
  

  • Researcher-made: must be testes for validity and reliability.
  • Adopted or Modified: should indicate its description as to its items, scoring, and qualification. Additionally, ask for a consent from the original author or researcher.
  • Copied
• The researcher must explain its parts and how the instrument will be validated. The instrument to be used should be appended (supported or supplemented).
• Survey Questionnaire: commonly used type of instrument in research that lists questions about the topic with spaces for responses.
• Types of questions in survey questionnaires:

  • Yes or No
  • Recognition (Recognizing the most appropriate answer)
  • Completion (Completing Words)
  • Coding (Points or Scales)
  • Subjective (Statement)
• A good questionnaire contain different types of questions:

  • Personal (Demographics)
  • Attitudinal or Behavioral
• Strategies for good questionnaires:

  • Clear Language
  • Questions or Answers DO NOT overlap
• A good questionnaire must be valid and reliable (underwent a pilot testing to a small number of individuals).
• Assessing instruments:

  • What instrument will you use to gather data to answer you researcher question?
  • Describe the instruments. Why did you choose these instrument? (Stating reasons or basis from other research)
  • What are the major parts of you instrument including the major variables and sub-variable to be used?
  • What rating scale will you use? (Likert scales)
• Reliability: consistency of results
• Validity: ability of an instrument to measure what it intends to measure (accuracy of the instrument)
• Types of Validity:

  • Content Validity: extent to which the content or topic of the test is truly representative of the content of the course.
  • Face Validity: visual appearance.
  • Construct Validity: the test measure a theoretical construct or trait. [if a student have high grades, does this means that he is smart]
  • Criterion Validity: degree to which the test agrees or correlates with a criterion set up as an acceptable measure.
  • Predictive Validity: how well the predictions made from the test are confirmed by evidence gathered at some subsequent time.
• Types of reliability:

  • Test-retest Reliability: administering the instrument twice in a same group.
  • Equivalent Forms Reliability: administering 2 identical in all aspects except the actual wording of items.
  • Internal Consistency Reliability: measure how well the items in two instruments measure the same construct.
  • Inter-rater Reliability: consistency of scores assigned by two or more raters on a certain set of results.
• Pilot Testing: preliminary administration of your instrument to a sample population to determine its effectiveness and identify areas needing improvement.
• Qualitative Research Instruments


  • Interviews
  • Observation
  • Free-Answer (Open-ended Question)
• Quantitative Research Instruments


  • Survey
  • Guided Response Type
  • Multiple-choice or Multiple Response Questions
  • Questionnaires
• Interviews or the interaction where verbal questions are posed by an interviewer to elicit verbal responses from an interviewee
• Observation (watching what people do) is a type of correlational (non-experimental) method where researchers observe ongoing behavior.
• Free-Answer: Also referred to as open-ended questions, these include unrestricted, essay, or unguided questions
• Survey research encompasses any measurement procedures that involve asking questions of respondents
• Guided Response Type: Recall-type questions asking the participant to recall a set of categories.
• A questionnaire is a research instrument consisting of a series of questions for the purpose of gathering information from respondents.
• Tables are generally used to present large amounts of exact values of qualitative or quantitative data, rather than quantitative information such as trends or patterns.