# 3.10: Non-Probabiltity Sampling

$$\newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} }$$

$$\newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}}$$

$$\newcommand{\id}{\mathrm{id}}$$ $$\newcommand{\Span}{\mathrm{span}}$$

( \newcommand{\kernel}{\mathrm{null}\,}\) $$\newcommand{\range}{\mathrm{range}\,}$$

$$\newcommand{\RealPart}{\mathrm{Re}}$$ $$\newcommand{\ImaginaryPart}{\mathrm{Im}}$$

$$\newcommand{\Argument}{\mathrm{Arg}}$$ $$\newcommand{\norm}[1]{\| #1 \|}$$

$$\newcommand{\inner}[2]{\langle #1, #2 \rangle}$$

$$\newcommand{\Span}{\mathrm{span}}$$

$$\newcommand{\id}{\mathrm{id}}$$

$$\newcommand{\Span}{\mathrm{span}}$$

$$\newcommand{\kernel}{\mathrm{null}\,}$$

$$\newcommand{\range}{\mathrm{range}\,}$$

$$\newcommand{\RealPart}{\mathrm{Re}}$$

$$\newcommand{\ImaginaryPart}{\mathrm{Im}}$$

$$\newcommand{\Argument}{\mathrm{Arg}}$$

$$\newcommand{\norm}[1]{\| #1 \|}$$

$$\newcommand{\inner}[2]{\langle #1, #2 \rangle}$$

$$\newcommand{\Span}{\mathrm{span}}$$ $$\newcommand{\AA}{\unicode[.8,0]{x212B}}$$

$$\newcommand{\vectorA}[1]{\vec{#1}} % arrow$$

$$\newcommand{\vectorAt}[1]{\vec{\text{#1}}} % arrow$$

$$\newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} }$$

$$\newcommand{\vectorC}[1]{\textbf{#1}}$$

$$\newcommand{\vectorD}[1]{\overrightarrow{#1}}$$

$$\newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}}$$

$$\newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}}$$

$$\newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} }$$

$$\newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}}$$

$$\newcommand{\avec}{\mathbf a}$$ $$\newcommand{\bvec}{\mathbf b}$$ $$\newcommand{\cvec}{\mathbf c}$$ $$\newcommand{\dvec}{\mathbf d}$$ $$\newcommand{\dtil}{\widetilde{\mathbf d}}$$ $$\newcommand{\evec}{\mathbf e}$$ $$\newcommand{\fvec}{\mathbf f}$$ $$\newcommand{\nvec}{\mathbf n}$$ $$\newcommand{\pvec}{\mathbf p}$$ $$\newcommand{\qvec}{\mathbf q}$$ $$\newcommand{\svec}{\mathbf s}$$ $$\newcommand{\tvec}{\mathbf t}$$ $$\newcommand{\uvec}{\mathbf u}$$ $$\newcommand{\vvec}{\mathbf v}$$ $$\newcommand{\wvec}{\mathbf w}$$ $$\newcommand{\xvec}{\mathbf x}$$ $$\newcommand{\yvec}{\mathbf y}$$ $$\newcommand{\zvec}{\mathbf z}$$ $$\newcommand{\rvec}{\mathbf r}$$ $$\newcommand{\mvec}{\mathbf m}$$ $$\newcommand{\zerovec}{\mathbf 0}$$ $$\newcommand{\onevec}{\mathbf 1}$$ $$\newcommand{\real}{\mathbb R}$$ $$\newcommand{\twovec}[2]{\left[\begin{array}{r}#1 \\ #2 \end{array}\right]}$$ $$\newcommand{\ctwovec}[2]{\left[\begin{array}{c}#1 \\ #2 \end{array}\right]}$$ $$\newcommand{\threevec}[3]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \end{array}\right]}$$ $$\newcommand{\cthreevec}[3]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \end{array}\right]}$$ $$\newcommand{\fourvec}[4]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}$$ $$\newcommand{\cfourvec}[4]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}$$ $$\newcommand{\fivevec}[5]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}$$ $$\newcommand{\cfivevec}[5]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}$$ $$\newcommand{\mattwo}[4]{\left[\begin{array}{rr}#1 \amp #2 \\ #3 \amp #4 \\ \end{array}\right]}$$ $$\newcommand{\laspan}[1]{\text{Span}\{#1\}}$$ $$\newcommand{\bcal}{\cal B}$$ $$\newcommand{\ccal}{\cal C}$$ $$\newcommand{\scal}{\cal S}$$ $$\newcommand{\wcal}{\cal W}$$ $$\newcommand{\ecal}{\cal E}$$ $$\newcommand{\coords}[2]{\left\{#1\right\}_{#2}}$$ $$\newcommand{\gray}[1]{\color{gray}{#1}}$$ $$\newcommand{\lgray}[1]{\color{lightgray}{#1}}$$ $$\newcommand{\rank}{\operatorname{rank}}$$ $$\newcommand{\row}{\text{Row}}$$ $$\newcommand{\col}{\text{Col}}$$ $$\renewcommand{\row}{\text{Row}}$$ $$\newcommand{\nul}{\text{Nul}}$$ $$\newcommand{\var}{\text{Var}}$$ $$\newcommand{\corr}{\text{corr}}$$ $$\newcommand{\len}[1]{\left|#1\right|}$$ $$\newcommand{\bbar}{\overline{\bvec}}$$ $$\newcommand{\bhat}{\widehat{\bvec}}$$ $$\newcommand{\bperp}{\bvec^\perp}$$ $$\newcommand{\xhat}{\widehat{\xvec}}$$ $$\newcommand{\vhat}{\widehat{\vvec}}$$ $$\newcommand{\uhat}{\widehat{\uvec}}$$ $$\newcommand{\what}{\widehat{\wvec}}$$ $$\newcommand{\Sighat}{\widehat{\Sigma}}$$ $$\newcommand{\lt}{<}$$ $$\newcommand{\gt}{>}$$ $$\newcommand{\amp}{&}$$ $$\definecolor{fillinmathshade}{gray}{0.9}$$

Suppose your friend were to tell you “candy is sweet.” You would probably accept the statement without argument, since we generally think of candy as almost a synonym for sweet in the U.S. However, if you wanted to prove the claim wrong and demonstrate that candy is not always sweet, you could conduct an experiment to see if candy is sometimes not sweet.

Given the practically limitless different types of candy, how could you collect a useable sample if you can’t possibly give each and every type of candy an equal chance of being a part of a random selection?

## Non-Probability Sampling

There are a number of recognized non-probability sampling methods, including:

• Convenience Sampling – Choosing samples based on easy or convenient access
• Volunteer (or Snowball) Sampling – Asking for volunteers or for recommendations from other samples
• Judgment Sampling – Deliberately choosing samples based on a desired characteristic
• Quota Sampling – Choosing samples to fill a specific quota of each of several sup-populations of the original

Some studies do not lend themselves to the collection of a randomized sample. Although a random sample is necessary if the goal is to directly generalize results back to an entire population, sometimes that isn’t the purpose of the study.

Common situations where non-probability sampling may be appropriate include:

1. Qualitative Research – Studies with the general goal of identifying topics worthy of future more detailed (quantitative) study
2. Studies particularly focused on specific portions of large population
3. Studies with limited funding and/or time
4. Studies with a goal of either disproving a particular theory, or of demonstrating the existence of a specific trait in a population.

### Determining Sampling Method Used

A toy designer is looking to develop the next ‘big thing’ in toys for young children. A brainstorming session results in 15 possible new ideas across a wide range of types from puzzles to remote-controlled chickens. A full-scale prototype development and testing for each idea would not be cost effective, so the company decides to perform a preliminary study. One of the employees suggests drawing sketches of the ideas and taking them to his son’s daycare to see which pictures get the most attention from the kids there.

What kind of sampling is this? Is it an appropriate choice for the application?

Solution: As a preliminary data collection resource, this convenience sampling of kids is a reasonable choice. Once the suggestions have been narrowed down a bit by judging the responses of the day-care kids, a more conventional random sampling may be used to pick out a specific design or two and then further refine it.

### Choosing the Most Appropriate Sampling Method

You are trying to convince your teacher that listening to music while doing homework improves scores. You decide to conduct a study of the effects of listening to music while doing homework and correlate it to student scores. Since your hypothesis is that listening to music improves scores on homework, would it be most effective to select a random sample of all the students in your school? If not, what type of non-probability sampling would be most appropriate and why?

If you are trying to find out if listening to music affects homework scores, then you would want a random selection of the entire population. However, since your very specific goal is to demonstrate music being associated with high scores, you might want to take a judgment sample of only students who listen to music while doing homework, and have high scores.

### Finding Missing Values

The student council at Cedar Valley Public School wants to gauge student opinion on the quality of their extracurricular activities. They decide to survey approximately 150 of the school’s 1,000 students using the grade levels (7 to 12) as the sub-population.

The table below gives the number of students in each grade level. Fill in the missing values for the number of students that should be in the sample from each grade level.

 Grade level Number of students enrolled Percentage of students (%) Quota of students in sample of 150 7 150 8 220 22 9 160 10 150 15 11 200 12 120 12 Total 1,000 100 150

First fill in the missing percentages by dividing the number of students in each grade by the total number of students in the school:

Now apply each grade’s percentage of enrollment to the sample size of 150:

• Grade 7 sample should contain 15% of 150 or 22.5 students, rounded up to 23 students
• Grade 8 sample should contain 22% of 150 or 33 students
• Grade 9 sample should contain 16% of 150 or 24 students
• Grade 10 sample should contain 15% of 150 or 22.5 students, rounded up to23 students
• Grade 11 sample should contain 20% of 150 or 30 students
• Grade 12 sample should contain 12% of 150 studentsor 18 students

### Earlier Problem Revisited

It is commonly accepted that any survey conducted by mail, or over the internet, or by telephone will have a very low response rate. It is not unheard of for such surveys to have less than 5% of the chosen sample actually return usable results.

It certainly seems logical to attempt to get a response rate as high as possible, but does a low response mean that the experiment is invalid?

No, it doesn’t. Certainly a very low response rate should be investigated, but the more important consideration is how well the results actually collected represent a random sampling of the population under study.

## Examples

### Example 1

Your mom says that sticking your tongue out every day will cause your face to get stuck that way. Would a non-probability sample be appropriate for a study attempting to prove that hypothesis wrong? What type would you recommend, and why?

This is actually an appropriate use of convenience sample. According to the hypothesis "...will cause it to get stuck", all you need is one example of it not getting stuck to disprove the statement. Since any example will work, you might as well try easy possibilities first.

### Example 2

Would a non-probability sample be appropriate for a study attempting to show that brand-name band-aids are superior? Which type would be appropriate and why?

No. Here you are hoping to generalize from your sample to the whole population of band-aid users. Extrapolation requires a true random sample.

### Example 3

How would a Snowball sampling method apply to a study of which flavor of gum has the longest-lasting flavor?

Here your study is actually on gum rather than the chewer, so asking friends to ask friends what gum they have found has the longest flavor would be convenient and appropriate

### Example 4

Would a non-probability sample be a good choice if you want to run a study to see if people who wear glasses are the best students? What type of sample would you recommend and why?

A judgment sample of the best students to see if they wear glasses would be a more efficient way to test your hypothesis than just a random sample of all students or all glasses-wearers.

## Review

1. A marketing company offers \$75.00 to the first 100 people who respond to their advertisement in a magazine and complete a questionnaire. This situation is an example of:

a. simple random sample

b. convenience sample

c. voluntary response sample

d. multistage cluster sample

The marketing class at a local high school wants to conduct a survey of the opinions of 60 students. Identify each type of sampling method they might use listed below.

2. Survey the first 60 students to walk through the doors at school in the morning.

3. Marketing class members each ask a friend for his/her opinion, and for the names of 3 other students to ask also.

4. Class members discuss who would be most appropriate to survey based on the results they want, then choose those persons.

5. Marketing class members decide to split students up into groups based on color of clothing, then choose a sample with the same ratio of colors as the whole school.

6. Number the students in the official school roster. Use a table of random numbers to choose 60 students from this roster for the survey.

7. A researcher plans a study to examine the depth of belief in an afterlife among the adult population of a small town. He obtains a simple random sample of 100 adults as they leave church one Sunday morning. All but one of them agree to participate in the survey. Which of the following is a true statement?

a. Proper use of chance as evidenced by the simple random sample makes this a well-designed survey.

b. The high response rate makes this a well-designed survey.

c. Selection bias makes this a poorly designed survey

d. None of these statements is true.

8. Do any of the following use simple random sampling?

a. Bingo game

b. Presidential elections

c. US Census

Identify the type or types of sampling used for the following.

9. Sarah went through a telephone directory and called every person with a name she liked.

10. Four people divided a telephone directory evenly and each called the first 10 numbers they found.

11. Every 5th block of 10 students walking past the classroom where the surveyors are working is exhaustively sampled about their faith in opinion polls.

Describe possible weaknesses of each of the following sampling procedures:

12. A sample of size 200 from a population of corporate personnel were asked to give their opinion about the federal government’s affirmative action hiring program. Ninety-three percent expressed opposition to the program.

13. A TV show takes a survey by asking individuals to call in to identify whether they are ‘FOR’ or ‘AGAINST’ more restrictions on gun control.

14. We are interested in obtaining a sample representative of all males age 18 or older. We run an advertisement on the Internet, ask for volunteers, and then choose a random sample from the list of people who volunteer.

## Vocabulary

Term Definition
convenience sampling Convenience sampling refers to the process of choosing a sample based on members who are easily accessible.
judgement sampling Judgment Sampling is a type of sampling occurs when the investigator already has made an assumption about a characteristic of the population, and samples are selected accordingly.
non-response bias Non-response bias is commonly caused by self-selection, subjects with a reason not to respond which may be unrelated to the actual study are not included, skewing the results.
quota sampling Quota Sampling is a type of non-random sampling where sampling is done until a specific number of subjects for various subpopulations have been selected.
volunteer (snowball) sampling Volunteer (Snowball) Sampling is a type of sampling requires that people volunteer themselves or their friends for a study.