Usable Security and Privacy
Problem Set 3

Due on Canvas at 10:45am on Tuesday, April 23rd.

Problem 1 (25 points)

Pretend that you are an IRB reviewer and you have received the experimental protocol described below.
Proposed protocol: We will follow a two-part protocol to study the usability of fingerprint readers on ATMs. This experiment is particularly timely since many Chicago-area banks have recently installed fingerprint scanners on their ATMs to make sure that only verified account owners can withdraw money. The first part will be an observational field study. We will conduct this field study at the First Bank of Blase branch on Ellis Avenue because there is a coffee shop located across the street. We will sit at an outdoor table at the coffee shop and watch everyone who goes to the ATM across the street. To make sure we don't miss anyone, we will also have a video camera at our table pointed at the ATM. The camera will be recording continuously. For each person who comes up to the ATM, we will record how many attempts are necessary for them to successfully authenticate to the fingerprint reader, as well as approximately how much money they take out. From the video recording, we will also estimate their height, weight, and ethnicity to see if those impact success using the fingerprint reader. To make it easier on our research team, we will crowdsource the estimation of height/weight/ethnicity by posting screencaptures on Amazon's Mechanical Turk platform and letting crowdworkers vote.

The second part of our study will be a between-subjects, in-lab experiment comparing the usability of different brands of fingerprint readers commonly used on ATMs. Participants will come to our lab, and we will begin by giving them a detailed demographics survey (age, occupation, annual income, sexual orientation, and past experience using biometric systems). Afterwards, they will use each fingerprint scanner on the market in randomized order. To understand the tolerance of the fingerprint reader at accepting partial matches of the fingerprints, we will retain participants' fingerprint readings from each device so that undergraduate students in a security class at our institution can analyze the tolerance as part of a class project. We will then administer a survey about participants' perceptions of the usability and comfort of each fingerprint scanner. To make sure our participants have enough free time to do our study, we will try to recruit students in grades 6-12 by posting flyers for the study in front of local schools. To reflect the amount of free time participants have, we will compensate students $5.00 for the study. Any non-student participants will receive $10.00 for the study.

1. Task 1-A: Create a bulleted list of potential ethics concerns raised by this protocol.
2. Task 1-B: Write at least one paragraph describing positive steps the researchers appear to have taken to minimize some of these ethics concerns.
3. Task 1-C: Write two paragraphs suggesting modifications to the study protocol that would better protect human subjects. The first paragraph should cover the first part of the protocol, and the second paragraph should cover the second part of the protocol.

Problem 2 (25 points)

Revelations from the past few years concerning widespread government surveillance have raised a number of questions about privacy and security. For example, have people started using Tor since hearing about government surveillance? What type of privacy tool do people most wish they had? Do attitudes about government surveillance differ between people from the US and people from India? Can average people determine what is or is not included in "metadata" about their telephone calls?

Pick any question of interest to you related to government surveillance and privacy. Although we strongly encourage you to come up with your own question, you may choose one of the questions above. Then, design and conduct a 9-participant survey on either Amazon's Mechanical Turk system or Prolific Academic to answer that question. You should turn in the following things:

1. Clearly state your research question and/or hypothesis in a single sentence.
2. Write one paragraph summarizing major decisions you made and things you kept in mind when designing your survey.
3. Include the text you used to recruit participants, as well as your screening criteria (e.g., "95% approval rating on MTurk"). If you use MTurk, note that MTurk assumes you only want "Master Workers" by default, which almost certainly will slow down data collection. To remove that requirement and/or add other requirements, click the tiny "Advanced Settings" text on the first page of the MTurk HIT creation page.
4. Include your survey questions, including answer choices for multiple-choice questions.

You may design and conduct this survey individually or with up to two partners (groups of 1-3 people total). If you work with partners, it is perfectly acceptable to submit the same recruitment text and survey questions. However, you should work by yourself to write the paragraph about designing your survey. Furthermore, while you will have the same raw data, you must complete the rest of this homework assignment individually.

Because this is for a class assignment and is not considered "research," you do not need to get IRB approval or include the UChicago consent form. You should still treat your participants ethically and pay them fairly for their time. You will need to spend your own money on this, but we don't anticipate this should cost you very much money, especially if you work in a group and split the cost. For fairness across groups, each group may spend at most $35 total on participant compensation (including MTurk fees) for this assignment.

Problem 3 (25 points)

Working by yourself, present the results of your study. Write at most one page describing the results, which should include at least one visual element (e.g., a graph, a figure, or a table). Note that you should not include the raw data itself in your submission. Instead, your results section should provide highlights and aggregate information, as in the papers you have read for this class.

Problem 4 (25 points)

Coordinating with your project teammates, discuss potentially related work for your class project and identify at least 20 research papers (4-person groups) or at least 25 research papers (5-person groups) that are related to your class project. Then, split these papers up such that each team member is responsible for five. For this homework, each of you should pick at least five papers that are different from what your teammates are picking, read them, and prepare the corresponding part of your related work section of the paper. Ideally, each teammate should choose a set of five papers that are closely related to each other.

For this homework, turn in the portion of the related work section of your final paper that discusses the five papers for which you are responsible.

Note that a related work section should not simply summarize each paper. Instead, you should connect the papers to describe what is known about the field. You should particularly note how your proposed project differs from or builds upon this prior work.

(CMSC 33210 only!) Problem 5 (0 points; -15 points if not completed)

Write a 3-7 sentence summary and short "highlights" for the Bravo-Lillo reading assigned for April 23rd.