A copy-edited version of this paper appeared as
Roth, Alvin E. "The National Residency Matching Program as a Labor Market," Journal of the American Medical Association, April 3, 1996, Vol. 275, No. 13, pages 1054-1056.
by Alvin E. Roth, Department of Economics, Harvard University, and Harvard Business School
March 1, 1996
The purpose of this article is to discuss what the NRMP does, why what it does is important, and what are the issues which will be the focus of the study of the NRMP I have been asked to direct.
Although medical students are unaccustomed to thinking of the NRMP as a market, as an economist I am struck by its similarities to other entry-level professional labor markets I have studied. What makes the NRMP unusual--although far from unique--is that the matching of residents to hospitals is organized via a computer algorithm. What makes it familiar are the problems the market experienced which led the NRMP to be organized this way. These problems mostly had to do with timing.
Let me begin with a capsule history of the market, extracted from Roth (1984). Then I'll discuss some other markets which presently have timing problems, and other markets which have employed matching algorithms, and why these sometimes fail. This will set the stage for a discussion both of why there is a great deal of common interest among students and hospitals in preserving an orderly market, and why there is also potentially room for disagreement between students and hospitals about what kind of matching algorithm should be employed. The study of the NRMP I have been asked to direct is intended to shed some light on this latter issue.
A capsule history:
Internships, the predecessors to today's residency positions, were introduced around 1900. One form which the competition among hospitals for interns took was that hospitals attempted to hire interns a little earlier than their principal competitors. Consequently, the date by which most internships had been finalized began to creep forward from the end of the senior year of medical school. Dates of appointment unravelled from one year to the next, first slowly then faster, so that by 1944 medical students were arranging their postgraduate employment as interns two years in advance of graduation. This meant that students had to apply for positions long before they were far enough along in their education to know their tastes and talents, and hospitals had to hire future staff with little information about how they would develop in their two remaining years of medical school. So there was considerable reason for dissatisfaction on all sides.
In 1945 the medical schools embargoed letters of reference, and the date of appointment was successfully moved to one year before employment would begin. In subsequent years the dates at which letters were released, and appointments made, were moved back into the senior year. But the problems in this market did not end when the appointment date was controlled. There followed a period in which students were called upon to make increasingly prompt decisions whether to accept offers. In 1945 offers were supposed to remain open for 10 days. Each subsequent year that interval was shortened, until by 1949 a grace period of 12 hours had been rejected as too long. What had happened was that hospitals found that, if an offer was rejected very near the deadline, it was often too late for them to reach their next most preferred candidates before they had accepted other offers. Even when there was a long deadline, much of this action was compressed into the last moments, because a student who had been offered a position at, say, his third choice hospital, would be inclined to wait as long as possible before accepting, in the hope of eventually being offered a preferable position. So the period before the deadline was frenzied, with students seeking to improve on the positions they had been offered by contacting the hospitals they preferred, and with hospitals sometimes pressuring students into early decisions in order not to have to contact students on their waiting lists after the deadline had expired. (Hospitals which applied such pressure gained an advantage over those which did not.) This congestion in the market, with its collateral missed opportunities and hasty agreements which were later sometimes not honored, led in 1952 to the use of a matching algorithm, in what was then called the National Intern Matching Program, the predecessor of the NRMP.
Some other markets with timing problems:
Table 1, from Roth and Xing (1994), lists several dozen markets and submarkets which have all experienced the unraveling of transaction dates that characterized the medical market before 1945 1. Table 1 concentrates primarily on professional labor markets. (But timing problems are not restricted to labor markets: note that the list leads off with the market for postseason college football bowls 2.)
| TABLE 1: A SELECTION OF MARKETS WITH TIMING PROBLEMS |
||
| Market
|
Organization
|
Stage
|
| Postseason college football bowls |
National Collegiate Athletic Association (NCAA) |
1 and 3 |
|
|
|
|
| Entry-level labor markets: |
|
|
| Federal court clerkships |
Judicial conferences |
2,1 |
| American law firms |
National Association for Law Placement (NALP) |
1 |
| Canadian articling positions |
Articling Student Matching Program |
|
| Toronto |
|
4 |
| Vancouver |
|
3 or 4 |
| Alberta (Calgary and Edmonton) |
|
3 |
| Entry-level business school markets: |
|
|
| New MBA's |
|
1 (occasionally) |
| New marketing professors |
|
1 |
| Other entry-level labor markets: |
|
|
| Japanese university graduates |
Ministry of Labor; Nikkeiren |
2 |
| Clinical psychology internships |
Association of Psychology Internship Centers |
2 |
| Dental residencies (three specialties and two general programs) |
Postdoctoral Dental Matching Program |
3 |
| Optometry residencies |
Optometric Residency Matching Services |
1 and 3 |
| Other two-sided matching: |
|
|
| Fraternity rush |
|
1 |
| Sorority rush |
National Panhellenic Conference |
3 |
| Entry-level medical labor markets: |
|
|
| American first-year postgraduate (PGY1) positions |
National Resident Matching Program (NRMP) |
3 |
| Canadian first-year positions |
Canadian Intern & Resident Matching Service |
3 |
| U.K. regional markets for preregistration positions: |
Regional health authorities |
|
| Edinburgh |
|
3 |
| Cardiff |
|
3 |
| Birmingham |
|
4,1 |
| Newcastle |
|
4,1 |
| Sheffield |
|
3 or 4, 1 |
| Cambridge |
|
3 |
| London Hospital |
|
4 |
| American specialty residencies: |
|
|
| Neurosurgery |
Neurological Surgery Matching Program |
|
| Ophthalmology
|
|
3 |
| Otolaryngology |
Otolaryngology Matching Program |
3 |
| Urology |
AUA Residency Matching Program |
3 |
| Urology |
AUA Residency Matching Program |
3 |
| Radiation Oncology |
Radiation Oncology Matching Program |
1 and 3 |
| Other specialties |
NRMP |
3 and 4 |
| Advanced specialty positions: |
|
|
| 12 (primarily surgical) specialties |
Specialties Matching Services |
3 |
| Three medical subspecialties |
Medical Specialties Matching Program |
3 |
| Four ophthalmology subspecialties |
Ophthalmology Fellowship Match |
3 |
| Plastic surgery |
Plastic Surgery Matching Program |
3 |
In these markets, too, considerable effort has been spent to halt and reverse the unraveling of transaction dates, and Table 1 lists for many of the markets the organization entrusted with this task. Some of these organizations, like the NRMP, were created for this purpose. Many of them have considerable compulsory power. But in many cases a solution to the timing problem has proved elusive. The difficulties encountered by these other markets may therefore illuminate some potential pitfalls we need to keep in mind when thinking about changes in the NRMP.
To make it easy to describe the common phenomena found in a diverse set of markets, Table 1 loosely categorizes each market as most recently being in one of four "stages," as follows.
Stage 1 markets are in the process of unraveling. Stage 2 markets have regulations specifying the time before which offers and sometimes other contacts cannot be made, and sometimes how long offers must remain open. But stage 2 markets are still decentralized, with employers contacting potential employees directly 3. Stage 3 markets have procedures which not only determine the timing of transactions, but also organize the transactions (e.g. the order in which offers are made, and at what point transactions are finalized). The most common form of stage 3 organization, like the NRMP, has employers and employees contacting each other (via applications, interviews, etc.) in a decentralized way, after which each employer submits a rank ordering of applicants to a central clearinghouse, and each applicant submits a rank ordering of positions. The clearinghouse uses these preference lists to produce a match according to some prespecified algorithm, and employers and employees are informed of the match. Stage 4 markets also use such centralized mechanisms, but have begun to unravel prior to the centralized market, for reasons and in ways similar to stage 1 unraveling.
The various markets for new law graduates in Table 1 show contemporary examples of both unraveling and centralized clearinghouses. The Table shows the same variety in contemporary regional medical markets in the National Health Service of the U.K. What makes the U.K. markets particularly informative is that, in the late 1960's and early 1970's, many of these markets instituted stage 3 matching algorithms some of which succeeded, and some of which failed (after which stage 1 unraveling resumed). In Roth (1991) (see also Roth 1990) it was shown that all of those which failed produced matches which were unstable, in the sense that there could be a student and hospital program who would both prefer to be matched to each other than to accept the matching produced by the algorithm. The algorithms failed in response to the stratagems such students and hospitals undertook to become matched to one another. In contrast, it was shown in Roth (1984) and Roth (1991) that (except for some problems with married couples) the NRMP, as it was then, and the two largest of the successful British algorithms, produced stable matchings, at which no such mutually unhappy students and hospitals exist. In general, there is considerable evidence that to successfully organize a market like the NRMP, the matching algorithm must produce a stable matching. Since the failures (and continued unraveling) in these markets are costly to all concerned, stable matchings are in everyone's interest.
But there can be more than one stable matching, and in general students and hospitals have room to disagree about which stable matching is the best for them. To get a quick idea of why this is so, consider a simple algorithm whose basic idea has been independently discovered in a number of markets (including the NRMP), but which was first mathematically understood, in an abstract setting, by Gale and Shapley (1962). In one version of this algorithm, the one on which the current NRMP algorithm is largely based, hospitals make offers to their highest ranked students, who tentatively hold the best of the offers they have so far received and reject the rest. Hospitals which are rejected at any stage make additional offers, to their next highest ranked students, and so forth, until no more offers or rejections remain to be made, at which point each student is matched to the position (if any) whose offer he or she is holding. Ignoring for a moment some of the complexities of the actual resident market (such as the fact that married couples and students who match to 2nd year positions require two positions), the matching produced in this way is stable.
However, a different stable matching in the simple market can be obtained by having the students make offers to the hospitals. To get an idea of why the hospital-offering algorithm is better for the hospitals and worse for the students than the student-offering algorithm, consider the special case in which the hospital-offering algorithm happens to end after the first round, i.e. in which each hospital makes offers to different students (on whose rank order lists they appear), so that no student issues any rejections. In this special case, each hospital would get its first choices, while each student would get the hospital which ranked him first, which of course might not be his first choice. In the reverse situation, with students making offers, if no hospitals issued any rejections then each student would get his first choice, and each hospital would get those students who ranked it first. So, e.g. in the hospital-offering algorithm, although a student may not do at all badly at the match which results, he might do better with the student-offering algorithm, and, furthermore, any matching at which he does worse cannot be stable. Of course the algorithms won't typically end after one round of proposals, but in the market with no couples or other complexities there remains a systematic advantage to the side which proposes.
The kind of mathematics used to analyze algorithms in this way is called game theory, and it not only speaks to questions of stability, but also of strategy. For example, we can ask if it is possible to design a stable matching algorithm with the property that, when it is used, no student or hospital can ever do better than to submit as his Rank Order List his full true preferences. This turns out to be impossible (Roth 1982, 1985; Sonmez 1996), although in a simple market without the complexities of the NRMP, the student-offering algorithm would at least have this property for the students. (A fairly comprehensive account of the mathematics is given in Roth and Sotomayor 1990.)
In a match such as the modern NRMP, which has some significant complexities, the story is not so simple--there may not be any stable matching (Roth, 1984), or even when there is there may be none which is systematically best for all hospitals, or for all students (Aldershof and Carducci, 1995). And no procedure for producing stable matchings is guaranteed to remove all strategic considerations for either side of the market. However there is good reason to believe that the properties found in simpler markets should carry over to the NRMP, at least to a large extent, at least most of the time. The study of the NRMP which is presently underway will allow us to make this last statement more precise.
Much of the recent debate (Williams; Peranson and Randlett 1995a,b; and AMSA/Public Citizen 1995) about what kind of matching algorithm the NRMP should employ has been phrased in terms of the theoretical results for the relatively simpler NRMP market analyzed in Roth (1984). Even aside from the complexities which have been introduced since that time, the theoretical results are better at telling us what must always happen and what can never happen than at telling us how often things which can possibly happen do in fact happen, and how large are their effects. We will need computational experiments, based on the data from recent matches, to tell us how large are the relative advantages given to hospitals or students by algorithms in which one side or the other makes offers, and how often a student or hospital can do better than to state true full preferences, and how much better can be done, at how much risk of being unmatched. These are the questions which will be the initial focus of the study now beginning. (Incidentally, the study proposal, Roth (1995), and a good deal of background material, is available on the world wide web at http://www.pitt.edu/~alroth/nrmp.html, and comments are very welcome.)
A more difficult set of questions will arise in any attempt to assess the extent to which the danger of renewed unraveling of the market may exist, either under the current NRMP algorithm or under a student-offering version, as the market changes in response to changes in health care financing. Ongoing monitoring of the health of the market would be a sensible precaution, whether or not any change in the algorithm is made.
Finally, note that in markets, confidence is a tangible but fragile thing. Just as loss of confidence in paper money causes hyperinflation, and loss of confidence in financial institutions causes bank runs, loss of confidence in a matching market like the NRMP can contribute to the unraveling of the market into a disorder which serves no one well. But confidence in a market is confidence in the behavior of others: in their willingness to accept paper money, in their ability to confidently leave their deposits in safekeeping, in their willingness to participate in the match in an orderly way. Once the technical results of the present study are available to inform the policy debate on how the NRMP should be organized, it will be the responsibility of all parties to conduct that debate in a way which does not unreasonably impair the confidence of either students or hospitals in the market which has served so long and so successfully at making order out of the disorder and inefficiency which often afflict professional labor markets. Whatever the outcome of the policy debate, it is in everyone's interest to preserve a thriving Match.
References:
Aldershof, Brian and Olivia M. Carducci [1995], "Stable Matchings with Couples," mimeo, Dept. of Mathematics, Lafayette College.
American Medical Students Association and Public Citizen Health Research Group [1995], "Report on Hospital Bias in the NRMP," http://pubweb.acns.nwu.edu/~alan/nrmp2.html [accessed 1/6/96].
Gale, David and Lloyd Shapley [1962], "College Admissions and the Stability of Marriage," American Mathematical Monthly, 69, 9-15.
Mongell, Susan J. and Alvin E. Roth [1991] "Sorority Rush as a Two- Sided Matching Mechanism," American Economic Review, 81, 441-464.
Peranson, Elliott, and Richard R. Randlett [1995a], "The NRMP matching algorithm revisited: Theory versus practice," Academic Medicine, 70, 477-484.
Peranson, Elliott, and Richard R. Randlett [1995b], "Comments on Williams' `A Reexamination of the NRMP Matching Algorithm," Academic Medicine, 70, 490-494.
Roth, Alvin E. [1982], "The Economics of Matching: Stability and Incentives," Mathematics of Operations Research, 7, 617-628.
Roth, Alvin E. [1984], "The Evolution of the Labor Market for Medical Interns and Residents: A Case Study in Game Theory," Journal of Political Economy, 92, 91-1016.
Roth, Alvin E. [1985], "The College Admissions Problem is not Equivalent to the Marriage Problem," Journal of Economic Theory, 36, 277-288.
Roth, Alvin E. [1990] "New Physicians: A Natural Experiment in Market Organization," Science, 250, 1524-1528.
Roth, Alvin E. [1991], "A Natural Experiment in the Organization of Entry Level Labor Markets: Regional Markets for New Physicians and Surgeons in the U.K.," American Economic Review, 81, 415-440.
Roth, Alvin E. [1995], "Proposed research program: Evaluation of changes to be considered in the NRMP algorithm," Consultant's report, http://www.pitt.edu/~alroth/nrmp.html.
Roth, Alvin E. and Marilda Sotomayor [1990], Two-Sided Matching: A Study in Game-Theoretic Modeling and Analysis, Cambridge University Press, 1990.
Roth, A.E. and X. Xing [1994] "Jumping the Gun: Imperfections and Institutions Related to the Timing of Market Transactions," American Economic Review, 84, 992-1044.
Sonmez, Tayfun [1996], "Manipulation via Capacities in Many-to-One Matching Problems," mimeo, Department of Economics, University of Michigan.
Williams, Kevin Jon [1995a], "A Reexamination of the NRMP Matching Algorithm," Academic Medicine, 70, 470-476.
Williams, Kevin Jon [1995b], "Comments on Peranson and Randlett's `The NRMP Matching Algorithm Revisited: Theory versus Practice'," Academic Medicine, 70, 485-489.
Endnotes
[Click on note numbers to return to the text at the note]
1. These are all what are called "two-sided matching markets." In some of them, the market determines both a matching and a set of wages. In others, including the NRMP, the market determines only the matching.
2. The unraveling of selection dates has even entered the language, in that recruitment by fraternities and sororities is now called "rush." The term arose because, by the latter part of the last century, entry into fraternities and sororities, which had initially been reserved for college seniors, had worked its way backward to the freshman class, and in some cases membership was arranged well before matriculation. The fraternities and sororities which went earlier than their competitors were said to be "rushing." See Mongell and Roth (1991).
3. The market for clinical psychology interns and postdocs is a stage 2 market which resembles the medical market between 1945 and 1951.