[BreachExchange] 'Massive' breach exposes hundreds of questions for upcoming SAT exams

[Inga Goddijn]

http://www.reuters.com/investigates/special-report/college-sat-security/

Shortly after David Coleman took over as CEO in 2012, the College Board
began redesigning its signature product, the SAT college entrance exam. The
testing company also hired a consultancy to identify the risks
associated with the monumental undertaking.

Among the red flags that consultant Gartner Inc raised in an October 2013
report: The not-for-profit College Board needed to better protect the
material being developed for the new SAT.

Plans to secure the new test from leaks or theft had “not been developed”
by the organization, the consultancy wrote in the report, reviewed by
Reuters. At risk were thousands of items, or questions, that were being
prepared for the redesigned SAT.

In 2014, employees at the New York-based College Board also raised
concerns, arguing for limits on who could access items and answer keys for
the revamped SAT, an email shows.

They were right to be worried.

Just months after the College Board unveiled the new SAT this March, a
person with access to material for upcoming versions of the redesigned exam
provided Reuters with hundreds of confidential test items. The questions
and answers include 21 reading passages – each with about a dozen questions
– and about 160 math problems.

Reuters doesn’t know how widely the items have circulated. The news agency
has no evidence that the material has fallen into the hands of what the
College Board calls “bad actors” – groups that the organization says “
<https://www.collegeboard.org/membership/all-access/counseling-admissions-financial-aid-academic/note-our-members-regarding-today>will
lie, cheat and steal for personal gain.”
<https://www.collegeboard.org/membership/all-access/counseling-admissions-financial-aid-academic/note-our-members-regarding-today>
But
independent testing specialists briefed on the matter said the breach
represents one of the most serious security lapses that’s come to light in
the history of college-admissions testing.

To ensure the materials were authentic, Reuters provided copies to the
College Board. In a subsequent letter to the news agency, an attorney for
the College Board said publishing any of the items would have a dire
impact, “destroying their value, rendering them unusable, and inflicting
other injuries on the College Board and test takers.”

College Board spokeswoman Sandra Riley said in a statement that the
organization was moving to contain any damage from the leak. The College
Board is “taking the test forms with stolen content off of the SAT
administration schedule while we continue to monitor and analyze the
situation,” she said.

Riley declined to say whether those steps would involve cancelling or
delaying upcoming tests. The next sitting of the SAT is October 1.

The breach is “a serious criminal matter,” Riley wrote. “A thorough
investigation is ongoing, therefore our comments must be limited.” The
College Board did not grant requests for interviews with CEO Coleman and
other employees named in this article.

The SAT is used by U.S. universities to help evaluate more than a million
college applicants a year, and so a major security lapse could cause havoc
for admissions officers and students alike.

That College Board security was breached is “a problem of a massive level,”
one that could “put into question the credibility of the exam,”  said Neal
Kingston, who heads the Achievement and Assessment Institute at the
University of Kansas.

If unscrupulous test-preparation centers were to obtain the items, the
impact on the SAT would be “devastating,” said James Wollack, director of
the Center for Placement Testing at the University of Wisconsin.

“Everyone will pull out all stops to try to compromise this test,” Wollack
said. That items for upcoming exams have leaked is “very alarming, very
concerning indeed.”

It’s too soon to know what damage the leak could cause. Harm can be
minimized if the items aren’t widely distributed. But Wollack and
Kingston noted that the latest incident is more serious than the widespread
SAT cheating reported in East Asia during the past few years.

As Reuters reported in March, the College Board has been unable to prevent
foreign test-preparation operators from
<http://www.reuters.com/investigates/special-report/college-sat-one/>giving
their clients an advance look at exam questions. Those problems were
primarily a result of the organization’s reuse of previously administered
exams. This breakdown involves test items that have never been given.

The materials provided to Reuters contain a wealth of items for upcoming
tests: reading passages drawn from novels, historical documents, scientific
journals, essays and other texts, each accompanied by questions. Also among
the materials were math problems involving geometry and quadratic equations.

The security breach comes as the College Board already faces pressure from
U.S. universities to better protect its marquee test.

The Reuters reports earlier this year detailed how an East Asian industry
<https://www.collegeboard.org/membership/all-access/counseling-admissions-financial-aid-academic/note-our-members-regarding-today>is
exploiting the College Board’s routine practice of recycling items from
past tests. Cram schools drill their students on questions harvested from
previous tests, conferring an enormous advantage over students who see the
items for the first
<http://www.reuters.com/investigates/special-report/college-sat-two/>time
when the exam is given.
<http://www.reuters.com/investigates/special-report/college-sat-two/>

In a statement at the time, the College Board pledged to do more to protect
the exam. University admissions officers, however, continue to voice
concerns to College Board officials
<http://www.reuters.com/article/us-college-sat-usa-idUSKCN0XH1PJ>about
reuse of exams. If the College Board can’t keep test material secure,
schools are left with the impossible task of determining whether an
applicant saw questions before taking the exam and therefore gained an
unfair edge.

UNEXPLAINED LEAKS

Questions about security inside the College Board emerged earlier this
year. Internal documents reviewed by this news agency showed that material
for past exams had been “compromised,” a term the College Board uses to
describe tests whose contents have leaked outside the organization.

In February, Reuters asked the College Board how it went about protecting
exam materials. Spokesman Zach Goldberg described the organization’s use of
lock boxes to help prevent the theft of SAT booklets sent to international
testing locations.

But lock boxes, he acknowledged, “would not preclude a leak that originated
earlier in the content development and distribution cycle.”

The question related to a confidential June 2013 PowerPoint presentation
the College Board prepared after a major security breach in South Korea.
After local test-prep operators obtained the test in advance, the College
Board canceled the May 2013 sitting in South Korea. The PowerPoint also
noted a type of breach that differed from the exploitation of recycled
tests: outright leaks of new test booklets.

According to the PowerPoint, SAT tests on specialized subjects – two in
Mathematics Level II and one in biology – had been compromised. These were
“new forms” – that is, tests that had never been administered in the United
States or abroad.

The PowerPoint gave no explanation for how those subject tests leaked. The
College Board has cautioned that “cartel-like companies” in China and other
countries “will stop at nothing to enrich themselves.”

EXAM INSECURITY

Historically, the development of questions to be used on the SAT was
primarily handled by the non-profit Educational Testing Service, or ETS.
Based in Princeton, New Jersey, ETS also oversees security for the College
Board when exams are administered.

After Coleman took over, however, the College Board began handling many
aspects of the SAT redesign in-house rather than through contractor ETS,
documents reviewed by Reuters show. The College Board also began managing
the “Item Bank,” the repository of questions created for the SAT. In the
past, that responsibility had belonged to ETS.

Taking on these roles gave the College Board greater control over the
material, internal documents show. Developing a single version of the
SAT typically takes about 18 to 30 months and costs about $1 million.

The College Board knew that assuming those roles presented challenges.

As its staff worked on the new exam in 2013, the outside consultant was
brought in to evaluate the risks the organization faced as it worked to
finish the redesign.

In an internal report from October 2013 labeled “FINAL DRAFT,” Gartner
advised the College Board to “develop and document a program security plan”
to handle test materials. The plan should address not only the physical
security of printed exam booklets but also the safeguarding of the College
Board’s network, servers, storage and data, the consultant recommended.

The security issues, the consultant concluded, presented a “medium” risk to
the College Board. A “medium” risk was defined as having “a potential
material impact...on program success that needs to be addressed proactively
at this time.”

Risks considered “high” included the issues related to the schedule and
budget for redesigning the test.

The report also recommended appointing a manager to protect the new exam.
It suggested the College Board “explicitly assign a Security Lead to the
Program with overall responsibility for all aspects of security related to
the Assessment Redesign Program and the redesigned assessments.” Officials
should “clearly document the responsibilities and mandate of this role.”

It’s unclear whether the College Board named a security chief or what
steps, if any, it took to protect exam materials stored digitally. In a
statement, spokeswoman Riley said the consultant later assessed how the
College Board responded to the recommendations and determined “we made
significant progress in every area, including our security policies and
procedures.”

A spokesman for consultant Gartner declined to comment about its findings
or recommendations.

An internal email shows that security concerns about access to test items
remained months after the consultant’s October 2013 report.

In a June 16, 2014 email to a College Board official, test development team
member Daming Zhu wrote that he and his colleagues  were concerned that too
many people inside College Board had “access to such secure data.” Zhu
helped manage the digital repository of items being developed for the new
SAT. The subject line of his email reads, “Secure Item/Test Information
Sharing.”

Zhu sent the email to Sherral Miller, vice president of assessment design
and development for the College Board.

Zhu’s worries were wide-ranging. “We are very concerned that IT is
duplicating key information of our items and test in a parallel system,” he
wrote. Another College Board unit also wanted exam information, Zhu
explained. He told Miller that “storing such important secure test data in
more than one place...is hard for us to understand.”

Zhu said the item bank team “believes that we ought to limit the access to
the secure item/test data, especially the [answer] keys, to the minimum
possible,” according to the email.

“Nowadays system hacking is not a surprise anymore,” Zhu wrote. “Expanding
the sources for secure test data will not help the security of test
information of high stake programs such as SAT…”

Zhu said the team “would appreciate some policies/guidelines from the
department or division upper management.”

Miller replied the same day, June 16. “You are right to be leery of them at
this time,” she said of the requests Zhu mentioned. Miller said she would
be discussing the matter with her boss “and will then get back to you so we
can set guidelines and policy.”

College Board spokeswoman Riley said the “reference to several internal
inquiries to access test item information” in Zhu’s email were “potential
scenarios that never manifested.” Riley said Zhu asked Miller “to confirm
College Board’s policies and guidelines in order to respond to these
inquiries, which Dr. Miller subsequently provided.”

Riley declined to share the guidelines, or to say how many College Board
employees and contractors had access to the test items.

Testing specialists said damage from the current breach can be limited so
long as the items aren’t widely distributed. They cautioned, however, that
major breaches have the potential to jeopardize the very existence of a
standardized exam.

“A test like the SAT … is so important and so consequential and is taken by
people all over the world,” Wollack said. The “College Board, especially
for this program, needs to be leading the industry in terms of security.”
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.riskbasedsecurity.com/pipermail/breachexchange/attachments/20160803/805d8557/attachment.html>