1. (4 pts). The lectures identified 3 fundamental questions to ask of any claim. Which of the following from that list were also said to be a focus of the lectures on data? MTF
A) How many data exist?
B) WhatÕs the evidence?
C) Who obtained the evidence?
D) How was the evidence obtained?
2. (5 pts.) Exam Key Code: Fill in (A)(B) on this question (2) to indicate your exam code; leave the other bubbles blank.
Also, fill in the correct bubbles for your name and pad number on the scantron form.
You must turn in this hard copy (with your name on it) and your scantron to receive credit for this exam.
Errors in Data
3. (4pts) The income of investment advisors increases with the number of clients they secure. The companies they represent often have hundreds of funds, many of which lost money in the previous year, but some of which earned money. Additionally, there are many different statistics calculated for their companyÕs overall performance, and some statistics are more favorable to the company than others. When an investment advisor meets with you, they show you the funds that did well and the statistics that are favorable to their company. Assuming that you can treat this difference between average statistic/fund performance and those shown to you as a type of error, what type of error is this? One answer only.
A) Sampling B) Bias C) RPA D) Human and technical E) None
4. (4pts) In a math class, you are asked to calculate the number of objects of length x (approximately an inch long) needed to span exactly one mile when laid out end to end. When attempting to do this in practice, you must measure the objects of length x to make the calculation. What type of error is most likely to cause your calculation to be somewhat inaccurate, assuming that the measuring device for the object gives a value only to within 1% of the true value? One answer only
A) Sampling B) Bias C) RPA D) Human and technical E) None
5. (4 pts) An employer decides to test his 200 employees for drugs. Using urine, he first checks creatinine levels to look for any diluted samples. He finds that, while nearly all of his employee samples have creatinine levels within the normal range, one employee has a low level, below 1% of that found in the population, and a second test of the same sample shows that the level is indeed low. Based on this evidence, he fires that employee for diluting their sample. What type of error could account for the unusually low creatinine level in one employee? (one only)
A) Sampling B) Bias C) RPA D) Human and technical E) None
6. (4 pts) The TAs for 301D report to Bull that many students attended the first help session for HW2, nearly all were prepared, and the students had lots of good questions and understood the material. From this, Bull concludes that everything is on track. Grading of HW2 instead reveals that most of the class was not as well prepared as the students in the first help session. What type of error causes the prediction based on the first help session to be inaccurate? one only
A) Sampling B) Bias C) RPA D) Human and technical E) None
7. (4pts) Which in the following list constitute(s) human and technical error? MTF
A) a thermometer that has lost some mercury and now reads 10¡ low
B) a piece of equipment whose display shows 2 decimal places but is often wrong at the third decimal
C) eyewitness mis-identification
D) failure to follow protocol
E) a person unconsciously feeling better because they think they took a pill that should make them better
8. (6 pts) A customer complains that prices charged at the Far West HEB grocery store are often different than the prices marked on the items, and further suggests that the difference often goes in favor of the store (the charged price is usually higher than the marked price). The store does an audit of all 3450 items it sells and finds that there are several items marked differently than charged. A distribution of the differences is shown below:
the center bar represents
items for which the charged price was the same as the marked price; the
bar goes up to 2250 bars to the left of
center are items in which charged price was less than marked, bars to the right of center are those in which
the charged price was more than marked.
Which options fairly describe the data and the customer complaints? MTF
A) The suggestion that the store usually charges more than marked is a claim that the price differences are biased.
B) The distribution shown in the figure does not support much of a bias (if any) in the price difference.
C) The chance of discovering bias would have been different had the 3450 items been chosen randomly.
D) The evidence for or against bias in prices charged versus marked depends on the height of the center bar
E) Any bias observed would have been eliminated by gathering the price data blindly
Ideal Data (Fixes and error)
9. (5pts) Which options identify a ÒfixÓ for the type of error indicated; a ÒfixÓ may either reduce that error or at least allow you to detect/measure that error. MTF
A) error: the lab sometimes commits sample mix-up. Fix: split the samples before testing; have each tested separately; look for discrepancies between identical samples
B) error: the lab deliberately creates ÔdataÕ that they think the prosecution wants. Fix: send requests to the lab blinded, so that the lab cannot discern what the prosecution wants
C) error: the lab calculates RMP improperly due to inadequate protocol Fix: split samples before testing; have each tested separately; look for discrepancies
D) error: lab occasionally declares false matches, but they go undetected. Fix: code the samples so that they are tested blindly, so that the lab does not know which samples should match
10. (5pts) The following explain how ÔfixesÕ for errors in data work. Which are correct? MTF
A) Standards donÕt necessarily correct H&T error, but they allow the determination of a H&T error rate. Once this rate is known, steps can be taken to reduce the error.
B) Blind fixes H&T error by concealing the purpose of the study, which in turn keeps a person from making mistakes
C) Better (more accurate) equipment is the main fix for RPA error, because most RPA error is due to the limits of measurements we can make.
D) Random is a fix for some types of bias. As demonstrated in class (with the Òchoose a random odd numberÕ demo), our minds exhibit unconscious biases, and truly random processes eliminate those biases.
E) Replication is a ÔfixÕ for bias. By obtaining many observations, it becomes possible to separate the bias from sampling error, more precisely knowing how much bias there is.
11. (5pts) The SFST has been validated in several U.S. studies with the following protocol (ÒofficerÓ means an SFST-certified officer).
i) Hundreds of drivers were tested by dozens of officers under normal (ÔfieldÕ) conditions
ii) Drivers were first given the SFST, then tested for their BAC (again in normal conditions)
iii) The officers were accompanied by trained observers and knew they were being observed
Overall, the SFST scores were found to be reliable indicators of BAC in these validation studies.
From these published validation studies, courts now argue that an SFST score by an officer when NOT OBSERVED is a reliable indicator of BAC. The following options address issues behind this assumption. Which are true? MTF
A) The validation tests were not done blindly because the officer knew he/she was being tested. Thus an officerÕs scoring of the SFST in the presence of an observer may be biased compared to a scoring in the absence of an observer.
B) The validation tests were not done on drivers chosen randomly. Thus an officerÕs scoring of the SFST in the presence of an observer may be biased compared to a scoring in the absence of an observer.
C) A bias due to officers being observed would be indicated if the SFST scores for a given BAC were consistently lower in the validation study than when observers were absent.
D) A bias due to officers being observed would be indicated if the SFST scores for a given BAC were consistently higher in the validation study than when observers were absent.
12. (5pts) A teacher grades written assignments while knowing the writersÕ identities. The grading may thus be subject to bias according to the teacherÕs preferences for certain students. Which of the following would be ways of greatly reducing or of demonstrating this bias (in case the teacher refuses to believe there is bias)? Assume that assignments are typewritten, so identification of hand-writing is not an issue. MTF
A) Grade multiple assignments from the same students. If some students consistently score low, then this reveals the bias in grading.
B) Have students put numbers on their assignments instead of names. The students donÕt tell the teachers their numbers until after the grading. This plan would eliminate the bias, rather than merely demonstrating it.
C) Have the teacher grade the assignments in random order. This randomization will substantially (though not completely) reduce the bias.
D) Compare exam scores to written assignment scores; exams are multiple choice and graded by a machine, so cannot be biased. If exam scores are consistently higher or lower than assignment scores for some students but not others, this illustrates the bias.
13. (5pts). You decide to test whether women prefer the taste of light or dark chocolate. To avoid confounding effects of the visual appearance, you conduct the tests in the dark, so that the women cannot see what they are tasting. Each of several women is given two pieces of chocolate, the order and identity determined by a couple of coin flips. Thus some women get two pieces of dark, some get two pieces of light, and some get one of each. They are then given the options of whether they prefer one over the other or have no preference. MTF
Mark the design features present, but only if it is explicitly present at some level in the problem description.
|
(A) explicit protocol |
(C) standards |
(E) blind |
|
(B) replication |
(D) random |
(F) none |
(14, 15). Do-it-yourself protocol. You are conducting an external review/test of a genotyping lab. Your job is to send two tubes to the lab, with labels. There are several options for the content of and label on a tube. You must decide which contents to send and how to label the tubes so that the features of ideal data requested in the question are present from the lab's perspective. If a tube has a person's name on it, the lab can assume that the tube contents belong to the name of the person on the label. If a tube is labeled with a number, the contents are unknown to the lab but known to you. A question mark (?) indicates that you do not know the individualÕs status for that characteristic. Your options for tube contents and tube labels are:
|
option |
tube label |
Contents in the tube are from |
Blood type |
Gender |
Marker status |
|
(A) |
Laura Baker |
Laura Baker |
AB |
Female |
+ |
|
(B) |
Darin Rokyta |
Darin Rokyta |
(?) |
Male |
(?) |
|
(C) |
Rachael Springman |
Rachael Springman |
A |
Female |
+ |
|
(D) |
#132 |
Darin Rokyta |
(?) |
Male |
(?) |
|
(E) |
#218 |
Patsy Cline |
(?) |
Female |
(?) |
|
(F) |
#10 |
Pam Hines |
O |
Female |
negative |
|
(G) |
Jerry Allison |
Jerry Allison |
O |
Male |
+ |
|
(H) |
#101 |
Brent Iverson |
A |
Male |
negative |
|
(I) |
No combination of tubes can satisfy the protocol |
|
|||
In the following questions, choose two letters among options (A)-(H) to describe the two tubes that will be sent to the lab. The tube labels are the only information the lab receives about the samples, and the lab does not have prior information about the individuals. If it is possible to satisfy the protocol, the question will require exactly two letters and only two letters -- one for each tube. Thus, the answer for a question might be (A) & (B), or it might be (D) & (F). If more than one pair of options are possible correct answers, fill in only one correct pair of options. Thus, if (A) & (B) is one acceptable answer, and (C) & (D) is another acceptable answer, fill in either (A)&(B) or (C)&(D), but not both. If a factor (such as identity, blood type, gender, etc.) is not specified in the protocol, then that factor will be ignored in grading the answer.
Alternatively, if a protocol cannot be satisfied with two from (A)-(H), fill in (I).
14. (4 pts) Choose two tubes to achieve replication of gender but nothing else. You should know both that gender is replicated and that nothing else is replicated, and the replication should be blind to the lab (you can assume the lab will know gender from the name on the tube).
two tubes or I: (A) (B) (C) (D) (E) (F) (G) (H) (I)
15. (3 pts) Choose two tubes to achieve replication of individual, marker, and blood type but gender is not replicated; the replication of individual should be blind to the lab Ð that is, the lab should not be able to tell from the information on the tubes that the two samples have the same blood type.
.
two tubes or I: (A) (B) (C) (D) (E) (F) (G) (H) (I)
Drug Testing, DWI testing
16. (4pts) The Ôscore sheetÕ shown in class for a DWI police report included which of the following features of ideal data? MTF
A) replication of the SFST test at several levels: multiple SFST tasks and multiple criteria scored within each of the SFST tasks
B) standards in the SFST by comparing the driverÕs performance with that of a known sober person at the same location
C) replication of BAC measurements
D) standards in the BAC measurements
E) randomization: the different SFST tasks are presented in random order
Criminal Justice System (includes DNA)
17. (5pts) You were given a question in class about how to reconcile the random match probability (RMP) with lab error rate (LER). Specifically, the question was what is the chance that the sample did not come from the suspect when you know the RMP and the LER? Which of the following are true as regards this problem? An option can be correct if it correctly describes something relevant to this question, even if it does not describe the chance that the sample did not come from the suspect. For purposes of illustration, you may assume the lab error rate is 1% and the RMP is 1/billion. MTF
A) Both the LER and RMP represent different reasons that the sample may not have come from the suspect. They both must be combined to obtain the overall possibility that the match is Ônot real.Õ It is always true that the overall chance the sample did not come from the suspect is at least as large as the larger of the RMP and LER.
B) The chance that the sample did not come from the suspect always lies between the RMP and LER.
C) You must decide the two issues separately, not together. Only once you know for sure the lab made an error can you worry about the error RATE.
D) Lab error rates can be estimated from proficiency tests
E) The RMP already includes the LER, so you only need the RMP.
F) The LER must always be larger than the RMP
18. (5pts) An eyewitness video was shown in class in which a single young male was observed. Following that video, the individuals in the class were asked to identify that person in a line-up. Which of the following is/are true about that demo? MTF
A) After viewing the video, the multiple choice options given to one section (e.g., 10:00) consisted of only the 6 people in the line-up. The other section was given those 6 plus the choice that ÔnoneÕ of the 6 people in the line-up was the one seen in the video. When given the choice of Ônone,Õ approximately half the class chose it.
B) Among those who chose one of the people in the line-up, there was approximately 90% agreement on who it was.
C) At least half of the people in the line-up were avoided by everyone in the class/section. That is, there were several people in the line-up for whom everyone in class agreed was not the person seen in the video.
D) The book and lecture suggested that the level of mis-identification demonstrated by our class is NOT typical of eyewitness ID in general.
19. (5pts) Lecture listed general properties that were ideal for a method used to identify individuals. Which of the following are true? The property must be one that we identified, and the benefit described must correctly apply. MTF
A) A reference database should exist for the method. A reference database allows one to calculate the lab error rate.
B) Labs/people performing the method should consistently pass proficiency tests. Passing proficiency tests provides assurances of the accuracy of the method.
C) The characteristics measured should be quantitative/continuous, not all-or-none (not discrete). These quantitative characteristics enable us to measure things very precisely, getting as close to the true value as we desire.
D) Independent verification of sample tests should be possible. Independent verification enables correction of individual cases of human and technical error.
E) All assays should be conducted by a single lab. Having one lab do all the tests ensures that the tests are done uniformly.
20. (5pts) For which of the following methods were reference databases said to exist (be used) in the U.S.? MTF
A) fingerprints
B) DNA typing
C) hair matching (by microscopic methods, not DNA)
D) polygraph
E) eyewitness ID
F) dog sniffing
21 (4 pts). The goal is to determine lab human and technical error rates of DNA typing through replicated typing of the same individuals. Ten tubes whose DNA type is unknown are sent to the lab. These 10 samples are from 8 people, but the lab does not know the number of people: every tube has a different code. For this goal, is the replication blind to the lab performing the typing? Why or why not? MTF
A) It is blind, because the lab does not know which tubes belong to the same person.
B) It is not blind regardless of the labels, because the lab can figure out which samples are the same after they do the DNA typing.
C) It is ambiguous as to whether the procedure is blind, because there are ways in which the procedure is blind and other ways in which it is not.
D) It is not blind because there is no standard sent with the tubes.
Data presentation
(If you were not in class, you can answer most of these options from the book.)
22. (4pts) The class responded differently to the presentation of absolute versus relative risks in a question concerning the benefits of a drug to reduce heart attack rates. Which are true?
A) Relative risks (as presented) can be used to inflate the benefit of a treatment by concealing the potentially large number of people who are not affected by the condition being treated.
B) Relative risks present slightly more information than absolute risks
C) The class responded somewhat more favorably toward presentation of a relative risk than of an absolute risk.
23. (4pts) In a similar vein as absolute vs. relative risks, there was a question asking for the calculation of the chance that a positive lab test was in error. Which are true or were claimed in the lecture?
A) The data to answer the question were given in either of two forms: numbers per 100 or percentages (natural frequencies vs. conditional probabilities).
B) Providing the data in relative terms (percentages) enabled more students to calculate the correct answer
C) It was mentioned that health professionals are mostly unable to make the correct calculations when the numbers are presented as percentages.
D) It is possible to make the same calculations with both types of data.