← return to practice.dsc80.com

Instructor(s): Janine Tiefenbruck

This exam was administered in-person. The exam was closed-notes, except students were allowed to bring two double-sided handwritten notes sheets. No calculators were allowed. Students had 180 minutes to take this exam. Questions 1, 2, 8, 9, and 10 are marked with an M; these test midterm material and were used for the redemption opportunity.

In this exam, you will work with a dataset of medical appointments at UC San Diego Health, to try to predict the amount of time patients had to wait for their appointments to start.

In the DataFrame med, each row represents a single medical appointment attended by a patient (no-shows are not included). The columns are:

• "MRN" (str): Medical record number, a unique identifier for the patient within the UC San Diego Health system.
• "Age" (int): The age of the patient.
• "Department" (str): The medical department where the appointment took place.
• "Provider" (str): The medical provider (doctor, or similar) for the appointment.
• "NumProviders" (int): The number of medical providers working in that department at the time of the appointment.
• "AppointmentTime" (pd.Timestamp): The time at which the appointment was scheduled to begin, using a 24-hour clock. Ends in one of :00:00, :15:00, :30:00, :45:00.
• "ArrivalTime" (pd.Timestamp): The time at which the patient arrived, to the nearest minute. Patients may arrive before or after their scheduled appointment time.
• "StartTime" (pd.Timestamp): The time at which the appointment actually began, to the nearest minute. The start time is always at or after the arrival time and the scheduled appointment time.

There are no missing values in med. The first five rows of med are shown below, though med has many more.

Throughout the exam, assume that we have already run the necessary import statements.

Problem 1

To start, we need to calculate patient wait times, which are not provided in our data. Suppose we execute the line of code below to add a "WaitTime" column to med.

med["WaitTime"] = (med["StartTime"] - med[["ArrivalTime", "AppointmentTime"]].max(axis=1)).dt.seconds / 60

Note that when we subtract two pd.Timestamp objects, the result is a pd.Timedelta object, whose .seconds attribute gives the time difference in seconds. There is no way to access the time difference in minutes directly.

Problem 1.1

Fill in the blanks in the code below so that the "WaitTime" column remains exactly the same as calculated above. In other words, the code below should give an equivalent way to calculate "WaitTime".

def wait_time(x):
    return __(a)__
med["WaitTime"] = med.apply(__(b)__)

1. What goes in blank (a)?

2. What goes in blank (b)?

Problem 1.2

What is the data type of the "WaitTime" column?

• pd.Timestamp

• pd.Timedelta

• int

• float

Problem 1.3

Determine the value of the following expression.

list(med["WaitTime"].iloc[:5])

Problem 1.4

What kind of values can appear in the "WaitTime" column? Select all that apply.

• Negative

• Zero

• Positive

Now that we’ve added a "WaitTime" column to med, we’ll also add a "Wait" column containing int values, defined as follows.

med["Wait"] = (med["WaitTime"] > 0).astype(int)

For the rest of the exam, med has "WaitTime" and "Wait" columns.

Problem 2

Doctors love having letters after their names! These letters usually represent degrees, titles, or certifications. We’ll refer to them collectively as credentials. In the "Provider" column of med, each provider has at least one credential. Credentials appear after the name and are separated by commas. For example, the preview of med shows that Dr. Takashi Hirase has two credentials (MD and MPH).

Problem 2.1

Write one line of code that evaluates to a Series containing the number of credentials for each provider in the "Provider" column of med. You must use .split() and you may not define any lambda functions.

Problem 2.2

Write a different single line of code that evaluates to the same Series. This time, you may not use .split() and you may not define any lambda functions.

Finally, we’ll add this Series to med as a new column called "Credentials". This column is included in med for the rest of the exam.

Problem 3

Consider the small subset of med shown in full below. Recall that the "Wait" column was added after Question 1. The data is sorted by "Age".

Problem 3.1

If we train a decision tree on this data to predict "Wait" based on "Age" and "NumProviders", what is the maximum possible accuracy the decision tree could achieve? Give your answer as an exact decimal or simplified fraction.

Problem 3.2

Select the expression below that gives the weighted entropy associated with using "Age" >= 15 as the root node of the decision tree.

• \frac{1}{6}\left(-\frac{2}{5}\log_2\frac{2}{5} - \frac{3}{5}\log_2\frac{3}{5}\right)

• -\frac{2}{5}\log_2\frac{2}{5} - \frac{3}{5}\log_2\frac{3}{5}

• \frac{1}{6}\left(-\frac{1}{2}\log_2\frac{1}{2} - \frac{1}{2}\log_2\frac{1}{2}\right)

• -\frac{1}{2}\log_2\frac{1}{2} - \frac{1}{2}\log_2\frac{1}{2}

• \frac{5}{6}\left(-\frac{2}{5}\log_2\frac{2}{5} - \frac{3}{5}\log_2\frac{3}{5}\right)

Problem 3.3

All but one of the following questions splits the data in such a way that the weighted entropy is the same. Which question yields a different weighted entropy than the others?

• "NumProviders" <= 2

• "NumProviders" <= 3

• "NumProviders" <= 4

• "NumProviders" <= 5

• "NumProviders" <= 6

Problem 3.4

What is the weighted entropy associated with any one of the questions you did not pick in part (c)? Give your answer as an exact decimal or simplified fraction.

Problem 4

sklearn is considering adding a new hyperparameter to its DecisionTreeClassifier class. The new hyperparameter, min_entropy, is used to determine when a node should be split. A node will be split when its entropy is greater than or equal to min_entropy. Otherwise, the node will be a leaf node.

Suppose we create training and testing datasets as follows.

X = med.drop(columns=["Wait"])
y = med["Wait"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

Problem 4.1

The function below selects a value for min_entropy based on an input list of candidate values.

def find_min_entropy(candidates):
    highest_score = -1
    out = -1
    for min_e in candidates:
        dt = DecisionTreeClassifier(min_entropy=min_e)
        dt.fit(X_train, y_train)
        if dt.score(X_train, y_train) >= highest_score:
            highest_score = dt.score(X_train, y_train)
            out = min_e
    return out

What should the function return on an input list of [0, 0.2, 0.4, 0.6, 0.8, 1]?

Problem 4.2

Circle one word in each box: If we train a decision tree with the value selected in part (a) for min_entropy, the test accuracy will likely be ____ than the train accuracy due to ____.

1. higher / lower
2. underfitting / overfitting

Problem 4.3

Circle one word in each box: In general, increasing the value of min_entropy ____ bias and ____ variance.

1. increases / decreases
2. increases / decreases

Problem 5

In Lab 9, you learned about k-nearest neighbors regression. A related machine learning algorithm is k-nearest neighbors classification, in which predictions are made by finding the k points in the training data that are nearest to the point we are trying to classify. We predict the class that the majority of those k points belong to (similar to the way in which decision trees in a random forest vote on a prediction). In this problem, we’ll use the standard Euclidean (L_2) distance to measure the distance between points.

In this problem, we’ll try to predict "Wait" based on "Age" and "AppointmentHour", where "AppointmentHour" is the hour from the "AppointmentTime" column.

Problem 5.1

Suppose the training data consists only of the 25 points shown below. Determine the accuracy, precision, and recall of a 3-nearest neighbors classifier on this data.

1. Accuracy:

2. Precision:

3. Recall:

Problem 5.2

Now suppose the training data consists only of the 25 points shown below. Determine the accuracy, precision, and recall of a 3-nearest neighbors classifier on this data.

1. Accuracy:

2. Precision:

3. Recall:

Problem 5.3

Finally, consider a 15-nearest neighbor classifier, which has the same accuracy on both datasets. What is that accuracy?

Problem 6

We want to use linear regression to predict "WaitTime" based on

• "NumProviders",
• "Credentials" (from Question 2),
• "Department",
• and whether "AppointmentTime" is in the morning (before 12:00) or afternoon (12:00 or later).

We want to ensure that the coefficients are interpretable and can be used to determine the most impactful single feature in the model’s predictions.

Fill in the code below to fit an appropriate Pipeline to the data in med, which we will think of as our training data for this problem.

def hour(df):
    df.iloc[:, 0] = df.iloc[:, 0].dt.hour
    return df
X = med[["NumProviders", "Credentials", "Department", "AppointmentTime"]]
y = med["WaitTime"]
pl = ____
pl.fit(X, y)

There is only one blank in the code above, which should be filled with a capital letter corresponding to one of the answer choice options given below. This answer choice will have blanks of its own, which you should also fill in. Every time you use an answer choice, fill in the blanks in that answer choice with one of the following:

• a capital letter corresponding to an answer choice option,
• a string, or
• an integer.

Some answer choices will be unused. You should leave any blanks in those answer choices empty.

Answer choice options:

A. drop = 'first'

B. remainder = 'drop'

C. remainder = 'passthrough'

D. PolynomialFeatures(___)

E. StandardScaler()

F. Binarizer(threshold = ___)

G. CountVectorizer()

H. FunctionTransformer(hour)

I. OneHotEncoder(___)

J. LinearRegression()

K. ColumnTransformer([("one", ___, ["AppointmentTime"]), ("two", ___, ["Department"])], ___)

L. ColumnTransformer([("one", ___, [___]), ("two", ___, [___]), ("three", ___, [___])], ___)

M. make_pipeline(___, ___)

N. make_pipeline(___, ___, ___)

Problem 7

Suppose we derive a numerical feature "AppointmentTimeSeconds" which measures the
"AppointmentTime" in seconds since midnight. Then we use linear regression to fit a prediction rule of the form: \text{predicted }\texttt{"WaitTime"} = w_0 + w_1\cdot\texttt{"Age"} + w_2\cdot\texttt{"NumProviders"} + w_3\cdot\texttt{"AppointmentTimeSeconds"}

Consider each of the following changes to the model above, and determine which coefficients in the fit model may change. Select all coefficients that may change. Note that we are changing the original model each time, not stacking changes on top of one another.

Problem 7.1

Change "AppointmentTimeSeconds" to "AppointmentTimeMinutes", which is measured in minutes since midnight.

• w_0

• w_1

• w_2

• w_3

Problem 7.2

Remove the intercept term w_0.

• w_1

• w_2

• w_3

Problem 7.3

Add a new feature, which is 3\cdot\texttt{"Age"}+\texttt{"NumProviders"}.

• w_0

• w_1

• w_2

• w_3

Problem 7.4

Add a new feature, which is \texttt{"Age"}/\texttt{"NumProviders"}.

• w_0

• w_1

• w_2

• w_3

Problem 8

Problem 8.1

We suspect that some "Provider"s have longer "WaitTime"s than others. Fill in the blanks below to add a column to med called "EstimatedWaitTime" which contains the median "WaitTime" for appointments with the same "Provider".

med["EstimatedWaitTime"] = (med.groupby(__(a)__)[__(b)__]
                               .__(c)__(__(d)__))

Problem 8.2

We suspect that some "Department"s are frequently running behind schedule and may occasionally have very high wait times. Fill in the blanks below so the result is a Series, indexed by "Department", containing the 95th percentile of "WaitTime" for each "Department" in which at least 75 percent of appointments have a "Wait". If less than 75 percent of appointments in a given "Department" have a "Wait", the "Department" should not appear in the Series. Recall that np.percentile(x, 95) calculates the 95th percentile of x.

    (med.groupby(__(a)__).__(b)__(__(c)__)
        .groupby(__(d)__)[__(e)__].__(f)__(__(g)__))

(a):

(b):

(c):

(d):

(e):

(f):

(g):

Problem 9

Suppose we have access to another DataFrame that contains billing information. The rows are the same as in med, but there are only three columns, "MRN", "AppointmentTime", and "Billed". The "Billed" column contains the amount that the patient was billed for their medical services at the time of the appointment.

Problem 9.1

Suppose patients are billed for services at the time of their appointment, unless the services are very complex (such as a surgical procedure). For complex procedures, the "Billed" column is left empty, and patients are charged for services at a later date. In this scenario, what is the most likely missingness mechanism of the "Billed" column?

• missing by design (MD)

• missing not at random (MNAR)

• missing at random (MAR)

• missing completely at random (MCAR)

Problem 9.2

Now suppose we merge the billing DataFrame with med on "MRN" and "AppointmentTime". We want to do a permutation test at the 0.05 significance level to decide if the missingness mechanism of the "Billed" column is more likely MCAR or MAR dependent on "Department". Which of the following test statistics could be used for this permutation test? Select all that apply.

• difference of means

• absolute difference of means

• total variation distance (TVD)

• K-S statistic

• none of these

Problem 9.3

Suppose the p-value comes out to 0.03. What can we conclude? Select all that apply.

• The missingness mechanism is more likely MCAR than MAR.

• The missingness mechanism is more likely MAR than MCAR.

• The missingness mechanism is not MD.

• The missingness mechanism is not MNAR.

• None of the above is a valid conclusion.

Problem 9.4

Suppose additionally that on March 1, 2026, UC San Diego Health experienced a technical outage and all the billing data for that day was lost. Which imputation strategy is most appropriate if we want to make sure the mean and standard deviation don’t change much as a result of the imputation?

• mean imputation

• probabilistic imputation

• mean imputation, conditional on "Department"

• probabilistic imputation, conditional on "Department"

Problem 10

Dr. Zheng and Dr. Golder are two medical doctors at UC San Diego Health. They each create a DataFrame of patients they have seen in the last year. Suppose that these DataFrames are called dr_z and dr_g and that each DataFrame includes a "MRN" column, which uniquely identifies patients.

Consider each of the following scenarios describing the overlap of dr_z and dr_g, and in each scenario, determine the number of rows in the DataFrame created by merging dr_z with dr_g using inner, outer, left, and right joins.

dr_z.merge(dr_g, on="MRN", how=???)

Problem 10.1

• dr_z has 100 rows, all representing distinct patients.
• dr_g has 80 rows, all representing distinct patients.
• 20 patients appear in both DataFrames.

1. how = "inner":

2. how = "outer":

3. how = "left":

4. how = "right":

Problem 10.2

• dr_z has 50 rows, all representing distinct patients.
• dr_g has 15 rows, all representing distinct patients.
• All patients appearing in dr_g also appear in dr_z.

1. how = "inner":

2. how = "outer":

3. how = "left":

4. how = "right":

Problem 10.3

• dr_z has 60 rows, representing 30 patients each appearing twice.
• dr_g has 80 rows, representing 40 patients each appearing twice.
• There are 10 patients that appear in both dr_z and dr_g.

1. how = "inner":

2. how = "outer":

3. how = "left":

4. how = "right":

Problem 11

Suppose we train a unigram, bigram, and trigram model on the following corpus.

corpus = "Patient is ill. Patient is in pain. Ill patient will recover 
          in time."

We tokenized the corpus as follows before training our models.

corpus.lower().split()

Now, we’d like to determine the probability of generating the sentence below, according to each model.

"Patient is in time."

For each part, give your answer as a simplified fraction (preferred) or a product of simplified fractions.

Problem 11.1

Determine the probability of the sentence above, according to the unigram model.

Problem 11.2

Determine the probability of the sentence above, according to the bigram model.

Problem 11.3

Determine the probability of the sentence above, according to the trigram model.

Problem 12

On the website for UC San Diego Health, each provider has their own page. We’ve scraped the HTML from one provider’s web page, which you can find below. We then instantiated a BeautifulSoup object, soup, from this HTML.

<html lang="en">
<head>
<link href="/assets/static/heroData-BGejBwUx.css" rel="stylesheet"/>
<title>Caitlin MacMillen, DO | Primary Care, Family Medicine, Osteopathic Medicine | UC San Diego Health</title>
<meta content="width=device-width, initial-scale=1, maximum-scale=5" name="viewport"/>
<meta content="Caitlin MacMillen is a Physician in San Diego with UC San Diego Health and specializing in Osteopathic Manipulative Treatment (OMT), Women's Health, Care for All Ages, Comprehensive Care for the Individual and Family, Overall Health and Well-Being, Family Planning." name="description"/>
<meta content="32.875663,-117.2133647" name="geo.position"/>
<meta content="San Diego,CA" name="geo.placename"/>
<meta content="US-CA" name="geo.region"/>
<script>window.yextAnalyticsEnabled=false;window.enableYextAnalytics=()=>{window.yextAnalyticsEnabled=true}</script>
<script type="application/ld+json">{
  "@context": "https://schema.org",
  "@graph": [
    {
      "@type": "Physician",
      "@id": "https://providers.ucsd.edu/details/33243/primary-care-family-medicine-osteopathic-medicine",
      "name": "Caitlin MacMillen, DO",
      "usNPI": "1518321629",
      "telephone": "(858) 657-8600",
      "isAcceptingNewPatients": false,
      "url": "https://providers.ucsd.edu/details/33243/primary-care-family-medicine-osteopathic-medicine",
      "knowsLanguage": [{"@type": "Language", "name": "English"}],
      "knowsAbout": ["Osteopathic Manipulative Treatment (OMT)", "Women's Health", "Care for All Ages", "Comprehensive Care for the Individual and Family", "Overall Health and Well-Being", "Family Planning"],
      "address": {
        "@type": "PostalAddress",
        "streetAddress": "9333 Genesee Avenue",
        "addressLocality": "San Diego",
        "addressRegion": "California",
        "postalCode": "92121",
        "addressCountry": "US"
      },
      "aggregateRating": {"@type": "AggregateRating", "ratingValue": 4.93, "bestRating": 5, "ratingCount": 188}
    }
  ]
}</script>
<script data-entity-id="16281938" id="yext-entity-data"></script>
<script async="" src="https://siteimproveanalytics.com/js/siteanalyze_14686.js"></script>
<script crossorigin="anonymous" src="https://kit.fontawesome.com/aa9c700570.js"></script>
</head>
<body>
<div id="reactele"></div>
</body>
</html>

Problem 12.1

Consider the DOM tree for this document. How many children does the root node have?

• 1

• 2

• 3

• 4

• 5

• none of these

Problem 12.2

What does the following line of code evaluate to?

len(soup.find_all("script"))

• 1

• 2

• 3

• 4

• 5

• none of these

Problem 12.3

The latitude and longitude for the provider’s office location are included in the document. Write one line of code that uses soup.find() (not soup.find_all()) to extract the latitude from soup, as a string ("32.875663").

Problem 12.4

You’ll notice that the HTML includes some JSON-formatted data. Locate the JSON object with keys "@context" and "@graph". Fill in the blank in the code below to read this JSON object in as a Python dictionary, dr_m.

dr_m_string = ___________
dr_m = json.loads(dr_m_string)
dr_m

Problem 12.5

Write one line of code that extracts the street address from dr_m, as a string.

Problem 13

The function re.match(pat, s) checks for the regular expression pat only at the beginning of string s. For example, re.match("o", "hello") does not find a match, but re.match("h", "hello") does.

Problem 13.1

The string "UC San Diego Health" has exactly two lowercase a’s. Write a regular expression pattern, pat, so that re.match(pat, s) finds a match if and only if s has exactly two lowercase a’s. Write clearly!

Problem 13.2

ICD-10-CM codes (International Classification of Diseases, Tenth Revision, Clinical Modification) are codes used in the medical field to classify diagnoses, symptoms, and causes of death. Below are a few examples of ICD-10-CM codes and their associated meanings:

• J45.909: Unspecified asthma, uncomplicated
• F32.9: Major depressive disorder, single episode, unspecified
• G20: Parkinson’s disease
• H43.391: Eye floaters, right eye
• S80.01XS: Contusion of right knee, initial encounter

ICD-10-CM codes consist of 3 to 8 characters following a certain format:

• Codes begin with a capital letter followed by two digits, which together specify the broad category of the medical condition.
• This may be followed by a decimal point and additional capital letters and numbers, which give more specific details about the medical condition.

Write a regular expression pattern, pat, so that re.match(pat, s) finds a match if and only if s is formatted like an ICD-10-CM code.

The following are some examples of incorrectly formatted codes that should not be matched.

• F27.
• TX3.120
• M27.56829
• L220.9

Write clearly!

👋 Feedback: Find an error? Still confused? Have a suggestion? Let us know here.