The Landscape of Modern Patent Appeals

The Landscape of Modern Patent Appeals

67 Am. U. L. Rev. 985 (2018).

*Professor, Ferguson-Carlson Fellow in Law, and Director of the Innovation, Business and Law Program, University of Iowa College of Law.  A project of this nature would not be possible without the support of many others.  Thanks to Alexander M. Zajicheck and Tyler W. Olson for their assistance in creating the Compendium interface; Rhonda DeCook and Louis Constantinou for their critical role in bringing this project to its current state; and Sarah Jack, Brett Winborn, Alex Lodge, and Rajul Patel for reading and coding the thousands of Federal Circuit decisions and helping to edit this Article.  Only routine research support and resources from the University of Iowa have been used in the creation of the Compendium to date.  Thanks also to David Schwartz, Paul Gugliuzza, Chris Seaman, and Mark Lemley (among many others) for feedback on the Compendium and earlier drafts of this Article.

Quantitative studies of the U.S. Court of Appeals for the Federal Circuit’s patent law decisions are almost more numerous than the judicial decisions they examine.  Each study painstakingly collects basic data about the decisions—case name, appeal number, judges, precedential status—before adding its own set of unique observations.  This process is redundant, labor-intensive, and makes cross-study comparisons difficult, if not impossible.  This Article and the accompanying database aim to eliminate these inefficiencies and provide a mechanism for meaningful cross-study comparisons.

This Article describes the Compendium of Federal Circuit Decisions (“Compendium”), a database created to both standardize and analyze decisions of the Federal Circuit.  The Compendium contains an array of data on all documents released on the Federal Circuit’s website relating to cases that originated in a federal district court or the U.S. Patent and Trademark Office (USPTO)—essentially all opinions since 2004 and all Rule 36 affirmances since 2007, along with numerous orders and other documents.

This Article draws upon the Compendium to examine key metrics of the Federal Circuit’s decisions in appeals arising from the district courts and USPTO over the past decade, updating previous work by scholars who studied similar populations during earlier time periods and providing new insights into the Federal Circuit’s performance.  The data reveal, among other things, an increase in the number of precedential opinions in appeals arising from the USPTO, a general increase in the quantity—but not necessarily the frequency—with which the Federal Circuit invokes Rule 36, and a return to general agreement among the judges following a period of substantial disuniformity.  These metrics point to, on the surface at least, a Federal Circuit that is functioning smoothly in the post-America Invents Act world, while also hinting at areas for further study.

INTRODUCTION
Statistics about the U.S. Court of Appeals for the Federal Circuit are plentiful.  Reliable, transparent, and clear statistics are not.   This Article and its accompanying database aim to change the status quo by providing a high quality, publicly accessible, and user-friendly source for quantitative data about the Federal Circuit.  Drawing upon this powerful database, named the Compendium of Federal Circuit Decisions   (“Compendium”), this Article provides key metrics to help answer questions about how the Federal Circuit is responding in the post-America Invents Act  world of patent law.
The Compendium was created to solve two problems plaguing empirical studies of the Federal Circuit.  First, almost every study of Federal Circuit decisions painstakingly recreates a basic set of data—case name, appeal number, judges, precedential status—before adding its own set of unique observations.   This process is redundant and labor-intensive, taking time and resources away from the actual focus of the study.   Second, variations in data sources and nomenclature among researchers make it challenging, and sometimes impossible, to conduct cross-study comparisons.   While researchers often desire to reproduce each other’s results or combine their data sets with previous ones to produce more complex analyses, the fact that researchers collect data from different sources and record it in different ways makes it difficult for subsequent researchers to perform these tasks.
The Compendium addresses these issues by providing a high-quality, well-documented data set together with a standardized data recordation framework.  That data set includes information about every document released on the Federal Circuit’s website—most notably, opinions and Federal Circuit Rule 36 affirmances.  Information about each document is recorded in a series of searchable fields, and data can be easily exported for further analysis.  In an effort to maximize transparency and encourage collaboration, this data set is available for future researchers to draw upon and, ideally, contribute to.  In addition, the Compendium is designed to simplify access and analysis for researchers who are not themselves involved in an empirical project but who wish to reference quantitative data about the Federal Circuit in a more robust way than through a query in a commercial database.   Consistent with principles of ethical legal research, its design is fully transparent.
In addition to describing the Compendium, this Article draws upon its contents to make several important observations about the Federal Circuit’s current decisions in appeals from district courts and the U.S. Patent and Trademark Office (USPTO).  In particular, while online news articles have provided some quantitative analyses of the Federal Circuit,  this study offers the critical infrastructure lacking in the popular legal press:  it publicly discloses the entire set of data underlying the observations, provides a detailed methodology for data collection and analysis, and rigorously examines the makeup of Federal Circuit decisions.
Analysis of the data reveals several important findings.  Since the passage of the 2011 America Invents Act, the number of Federal Circuit decisions in appeals from the USPTO has exploded, even as the number of decisions in appeals from district courts has remained relatively steady.   As the number of Federal Circuit decisions in appeals arising from the USPTO has grown, so too has the number of precedential opinions.   At the same time, however, the Federal Circuit has not resolved all of these new appeals through precedential opinions:  the lion’s share of resolutions continue to be through nonprecedential opinions and affirmances under Rule 36.   In other words, as other scholars note,  the Federal Circuit is resolving more cases with Rule 36 affirmances than ever before.  Importantly, however, the rate at which the court is employing Rule 36 affirmances has fluctuated within relatively limited bounds over the last decade.
Another aspect of the Federal Circuit’s decisions that the Compendium reveals is that, contrary to the trend of substantial disagreement among judges in the early 2010s,  Federal Circuit judges are now coming to unanimous agreement in precedential opinions more often and writing dissenting and concurring opinions less often.   This is a dramatic departure from a period when the average rate of precedential opinions including dissents exceeded the average rate of precedential opinions in which all the judges agreed.
Analysis of individual judges’ decisions reveals several notable patterns.  While all active judges participate in about the same number of Rule 36 summary affirmances, there is substantial variation in the number of precedential opinions authored by each judge.   And when it comes to precedential opinions, certain judges are notable for the high frequency of unanimous (i.e., joined by both of the other members of the panel) opinions they author,  while others—particularly Judge Timothy B. Dyk—are typified by writing precedential opinions joined by only one other member of the panel.
The remainder of this Article proceeds as follows.  Part I provides background on the Federal Circuit and previous empirical studies of the court.  Part II describes the data source, collection process, and data framework for the Compendium.  Part III presents descriptive statistics drawn from the Compendium, including numbers of Rule 36 summary affirmances over time and the extent of agreement among judges on the court.  Finally, Part IV draws some conclusions from these observations and offers some directions for future work.
I. BACKGROUND
This Part provides a brief introduction to the Federal Circuit and highlights a few of its aspects applicable to the Compendium.
A. The U.S. Court of Appeals for the Federal Circuit
The Federal Circuit is the Article III court tasked with hearing appeals in disputes involving certain types of substantive legal issues, including those related to patents.   At full capacity, there are twelve “active” judges on the court.   There may also be—and currently are—additional judges who have taken “senior” status.  Judges in senior status typically work about a quarter of the caseload of an “active” judge.   Occasionally, other judges will sit by designation.   Because the Federal Circuit’s appellate jurisdiction is defined by subject matter rather than the geographical origins of an appeal,  the Federal Circuit may decide cases arising from federal courts from California to New York, Texas to Minnesota—provided that the appeals fall within the court’s subject matter jurisdiction.  Practice before the court is governed by the Federal Rules of Appellate Procedure, as supplemented and modified by the Federal Circuit Rules of Practice.
The court hears appeals on a variety of subjects, including money claims against the government, trade issues, and appeals from the Court of Appeals for Veterans Claims and Merit Systems Protection Board.   Researchers studying patent law are principally interested in appeals involving patent issues.   These appeals primarily arise from the district courts and USPTO, although a few arise from the International Trade Commission and an even smaller number from the Court of Federal Claims.
Nearly all appeals to the Federal Circuit that arise from the district courts involve a dispute relating to a patent—typically a patent infringement suit based on 35 U.S.C. § 271.   The Compendium currently does not distinguish between appeals arising from the district courts that involve patents and those that do not.
Appeals from the USPTO come in two major flavors:  patents and trademarks.   Prior to September 16, 2012, appeals involving patent issues at the USPTO arose from the Board of Patent Appeals and Interferences (BPAI); since then, those appeals arise from the Patent Trial and Appeal Board (PTAB).   Appeals involving trademarks arise from the Trademark Trial and Appeal Board (TTAB).   As discussed in Part III, the number of Federal Circuit decisions involving trademark appeals from the USPTO remained under twenty per year, even as the number of decisions involving patents grew from about twenty in 2008 to nearly 200 in 2016.
Once at the Federal Circuit, appeals are assigned to a panel of three judges.   These judges read the briefs and preside over oral arguments.   After oral arguments, the panel of judges confers.   They affirm some appeals at this stage through application of Federal Circuit Rule 36.   These “Rule 36s” involve just one outcome:  an affirmance of the lower tribunal.   A Rule 36 affirmance requires agreement among all the judges on the panel, and the decision is not attributed to any single judge or group of judges.   Instead, the panel acts “per curiam,” or in unanimous agreement.
If the panel does not affirm through Rule 36, the most senior judge in the majority will assign a judge to write the opinion of the court.   Typically, the other members of the panel will join the court’s opinion; sometimes one or even both other judges will choose to write separately.   Although separate opinions are usually a dissent or a concurrence, judges occasionally write other categories of opinions.   The judges may also designate an opinion as “precedential,” making it binding precedent on future panels of the court.   If the judges do not designate an opinion as precedential, it remains nonprecedential, a status that limits its legal influence.
Appeals may also be resolved by orders of the court or settlement by the parties.   Because these results are not typically published to the Federal Circuit’s website, they are not included in the Compendium.
On occasion, the court may sit in panels of more than three judges or as a court en banc.   When the court issues an opinion en banc, that opinion has especially strong controlling weight and may only be overturned by the court again sitting en banc or by the U.S. Supreme Court.
Many scholars have written on the role of the Federal Circuit in patent law, and there are entire treatises devoted to the court.   Leading descriptive work on the Federal Circuit includes Rochelle Cooper Dreyfuss’s classic The Federal Circuit:  A Case Study in Specialized Courts   and more recent articles;  Paul R. Gugliuzza’s The Federal Circuit as a Federal Court;  Judge Pauline Newman’s The Federal Circuit:  Judicial Stability or Judicial Activism?   and Origins of the Federal Circuit:  The Role of Industry;  Laura G. Pedraza-Fariña’s Understanding the Federal Circuit:  An Expert Community Approach;  and Ryan Vacca’s extensive literature review, The Federal Circuit as an Institution.
B. Empirical Studies of the Federal Circuit
Given the array of theories about the role and function of the Federal Circuit, it is unsurprising that legal researchers and commentators have sought to assess or support those theories with empirical data about the court’s decisions.  Empirical studies of the Federal Circuit number in the dozens and take every shape and form imaginable, from glossy fliers  to detailed and methodical law review articles  to blog posts.   Ryan Vacca’s The Federal Circuit as an Institution is an excellent place to begin a foray into this area, as it provides an overview of every empirical study of the court through 2016.
A problem with these studies, however, is that they are not easy to compare.  This is not just because they examine different attributes of the court’s decision making, but also because they frequently measure the same thing using different systems of measurement.   Just as a Mars probe was once lost when one engineering group used English units of measurement while another used metric,  so too are problems presented by these varying ways of recording data about Federal Circuit decisions.
One example of this problem is discussed in a recent article examining the rate at which the Federal Circuit reverses district courts.   As that analysis demonstrates, the definition of the term “reversal” alone can affect rates by as much as 10%.   Other components of study design can also affect the observed reversal rate.   Difficulties with cross-study comparisons are not limited to reversal rates; virtually every empirical study of the Federal Circuit uses its own nomenclature, field descriptions, and data collection methodology.   Worse, sometimes these details are not provided in the study or accompanying documentation, ratcheting up the difficulty of understanding or comparing the study results.
Another problem with existing empirical studies is that they frequently rely on commercial databases that are designed to assist lawyers in conducting traditional legal research, not for empiricists seeking to maximize replicability and transparency.   The contents of these databases change over time, as do their interfaces.   Contractual limitations may also restrict what may be done with those databases.   The Compendium aims to reduce these barriers by providing a consistent and reliable source for empirical studies of the Federal Circuit using a standardized nomenclature and open access dataset.
II. METHODOLOGY
This Part describes the source of the Compendium’s data, the methodology used in its collection, and how the information it contains was recorded.
A. Data Source and Collection
1. The source of the decisions used to create the Compendium
The Compendium draws from the Federal Circuit’s own platform for releasing its decisions.   Using the Federal Circuit itself as a data source offers many advantages over other data sets, such as the U.S. Patent Quarterly, the Federal Judicial Center, Westlaw, or Lexis.   First, written decisions are published in their entirety on the website, rather than being condensed.   This allows researchers to extract a substantial amount of information about the decisions, including dissents, concurrences, and their respective authors.  Second, the website includes both precedential and nonprecedential decisions, thus providing an extensive collection of materials.   Third, the decisions collected from the Federal Circuit’s website are in the public domain,  and, unlike commercial databases, usage of the data is not restricted by contract.  Fourth, constructing a database based on records collected from the Federal Circuit allows it to be designed especially for use by academic researchers and other scholars of the Federal Circuit.  Consequently, the Compendium is structured to maximize reproducibility, transparency, and the types of information most useful to scholars of the Federal Circuit.
There are some important limitations on the Compendium that flow from its data source.  In particular, any data set is only as good as its source data.  Any conscious policy or inadvertent action that results in documents not being released on the court’s website means that those documents are not included in the Compendium.  Only three instances of this occurring are known to date.  The first is that although there are precedential and nonprecedential written opinions in the Compendium prior to 2007, there are no Rule 36 affirmances prior to 2007.  The second known data source limitation is that the Federal Circuit released a large number of orders on its website from 2010 to 2013, stopped doing so around 2013, and currently publishes only a handful of orders each year.   This limitation is considered of relatively minimal importance, at least for this Article, as it focuses on opinions and Rule 36 affirmances.  However, it is something to take into account for future studies drawing upon the Compendium.  The third is the smallest but potentially most concerning:  during the data verification process, the research team discovered that there was a period of time, September 2012 to March 2013, from which it appears that opinions and Rule 36 affirmances are missing from the court’s website.  The gap consists of eighty-nine precedential opinions, nonprecedential opinions, and Rule 36 affirmances that are not available on the court’s website.  Fortuitously, the seventy documents in appeals arising from the district courts were collected in 2013 as part of another project and are included in the database.   In addition, information about the nineteen decisions in appeals arising from the USPTO was collected from other sources and added to the Compendium.
Another limitation doesn’t relate directly to the completeness of the data source but does relate to its contents.  Occasionally, the Federal Circuit might issue an opinion in an appeal only to later withdraw it and issue a new one following a party’s request for rehearing.  The court may also change an opinion from nonprecedential status to precedential status after the initial opinion issued.  These events are relatively rare.  To the extent these documents were available on the court’s website during a collection period, they were collected and included in the Compendium.  In addition, a variety of deduplication methods were used to identify these situations and flag the earlier version as a duplicate.
Finally, because the Compendium is a database of “slip” opinions and orders released on the Federal Circuit’s website, the documents do not constitute the “official” versions published in the Federal Reporter.  That said, there are rarely major changes to an opinion once it is released and, when there are, those changes are accompanied by a new opinion.   The Compendium includes both original and revised opinions provided that they were available on the court’s website during a data collection period.
2. Assembly of the Compendium
The Compendium was constructed from documents released on the “Opinions & Orders” page of the Federal Circuit’s website.  Each record in the Compendium represents a single document posted to that page.   For example, record 10196 in the Compendium is the Federal Circuit’s en banc opinion in Phillips v. AWH Corp.   Because a record corresponds to a single document, an opinion or other form of decision that resolves multiple appeals consolidated by the court is treated as a single record.
In order to collect only documents in appeals arising from the district courts and USPTO, either “DCT” or “PATO” was selected from the “Origin” field on the Federal Circuit’s “Opinions & Orders” page.  Opinions were then downloaded by hand.
Each record contains multiple discrete pieces of information—or “fields”—that relate to the document.  During the collection process, coders recorded basic information—including the court of origin, the case name, and the appeal number—about the document provided on the Federal Circuit’s website through a copy-and-paste process so as to minimize coder error.   For record 10196, the “Case Name” was recorded as “PHILLIPS V. AWH CORPORATION, ET AL.,” the “Origin” as “DCT” (indicating that the appeal originated in the district courts), and the “Appeal Number” as “2003-1269.”  Additional information about the document was collected from the document itself.  For example, in record 10196 the “Document Type” field was coded as “Opinion,” and “En Banc” was recorded as “Yes.”
Due to the ongoing nature of this project, data were collected on multiple occasions.  An early set of the data was used in Disuniformity, a 2013 study of the rate of unanimity and dissents in Federal Circuit decisions arising from the district courts.   Additional data were collected over the period 2013 to 2017.  Due to the potential for error and variation in this collection process, it was subjected to an extensive verification process in 2017.
As of the end of 2017, there were 1477 records in the Compendium arising from the USPTO and 4397 records in the Compendium arising from the district courts, excluding duplicates.
B. Fields
The following attributes are recorded for each document:  (1) date; (2) appeal number; (3) origin; (4) case name; (5) precedential status; (6) document type; (7) en banc status; (8) judges; (9) opinion type (majority, dissent, etc.); (10) authorship; (11) URL; (12) notes; and (13) duplicate status.   The information for the first five categories was collected directly from the website, while research assistants manually coded information for the subsequent categories based on information contained in the document itself.  In addition, each record automatically received a unique record ID to make it easier to track and compare the data.  Recent work on the data set includes the classification of appeals by the specific tribunal of origin, such as the PTAB or TTAB, and the coding of outcomes.
Data were initially recorded in Microsoft Excel spreadsheets.  However, in the spring of 2017, the existing spreadsheets were combined and converted into a format accessible through a user-friendly application.   The new application includes a mechanism to quickly filter data based on field selection and permits exportation of filtered data to a .csv file for further analysis in a program such as Stata or Excel.  It also allows a user to view all of the fields for a given record at one time.  Working with a large spreadsheet in Excel—nearly 6000 records, each with dozens of fields—can be both time consuming and computer-power taxing, and the new application greatly simplifies interaction with the data.
To allow for even easier use of the data, access to the application is available via a website, accessible through https://empirical.law.uiowa.edu.   This version of the application allows users to filter, sort, and export the data in a version readable by Excel and other programs.  For security and stability reasons, the web-accessible version of the application is read-only and does not permit editing of the database itself.
Appendix A has a full sample record that lists the variables and format of an example opinion, Phillips v. AWH.   With the standardized set of fields and formatting, future researchers will be able to easily compare results using a single reference point.
Below is a summary of each field coded on the Compendium.  The Compendium itself also has an accompanying Codebook that provides further details about each field and addresses particular issues that arose during the coding process.
1. Case name
The case name is copied directly from the Federal Circuit website with no further abbreviation of parties or titles.  This helps eliminate errors in coding from one person to the next and therefore keeps the record consistent.  Whereas one researcher may keep “international” in the case name and another may abbreviate to “int’l,” the methodology used in the Compendium lets the Federal Circuit make that decision.  The case names also include the bracketed text found on the Federal Circuit’s website, such as “[OPINION]” or “[ORDER].” This further eliminates error as the case name in the database matches the primary source for the data.
2. Case date
The date is also directly copied from the Federal Circuit website.  This is the date the opinion was published in ISO 8601 form, or year-month-day.   While the format may not be the traditional way dates are recorded in the United States, ISO 8601 is the worldwide standard for “naming” dates and is the most frequent method for date recording in computer programming.   It is also the form used on the Federal Circuit’s website.   Therefore, copying directly from the Federal Circuit not only eliminates potential human error from format changing, but also allows computer-savvy researchers to directly input and code the output into their own databases.
3. Origin
The origin of the case is recorded directly from the Federal Circuit website and is identified by the Federal Circuit’s identification of the origin as either “DCT” or “PATO.”  This field was subjected to additional human verification, and a few rare errors in the Federal Circuit’s classification were corrected at that time.
4. Duplicate
Records that are identified as duplicates are flagged with “Yes” in the duplicate field.  A document is considered a duplicate if a record was inadvertently added to the database more than once or if it is a document that the Federal Circuit initially issued and then replaced.   In these situations, the earlier record is marked as the duplicate.
5. Precedential status
The precedential status is recorded as either “Nonprecedential” or “Precedential,” drawing directly from the Federal Circuit’s website subject to review by a human coder.
6. Document type
The document type was determined by examining the bracketed text in the case name, discussed above, and checking that information against information from within the document itself.  Possible document types are “Order,” “Opinion,” “Errata,” “Rule 36,” “No file,” and “Other.”   Note that while the earliest opinion released on the Federal Circuit’s website—and thus the earliest released on the Compendium—is October 13, 2004, the earliest Rule 36 affirmance is July 11, 2007.   The Federal Circuit did, however, affirm appeals prior to that date using Rule 36.   Consequently, while there were Rule 36 affirmances between 2004 and 2007, they are not contained in the data set.  “No file” indicates that the record entry on the Federal Circuit’s website did not have a document associated with it and the document type could not be otherwise determined from context.
7. En banc
En banc status is recorded for some Orders and for all documents identified as either “Opinion” or “Rule 36.”  The field is coded as “Yes,” “No,” or “Partial” based on information that is extracted from the document.  Occasionally, the Federal Circuit will issue an opinion that is en banc only in part.   The Compendium accounts for these opinions by coding the field as “Partial.”  As of the end of 2017, there were twenty-six opinions coded as “Yes” or “Partial” in the “En Banc” field.
8. Judge 1, Judge 2, and Judge 3
The last names of the first three judges on each panel are recorded in the fields “Judge 1,” “Judge 2,” and “Judge 3.”  These names are found at the beginning of the Federal Circuit document.   Typically, there are only three judges assigned to an appellate panel, and the order in which the document lists the judges is the order in which the fields were populated.  The Federal Circuit publishes judges’ names in uppercase font but the Compendium codes them in title case for readability.  In the rare instances where more than three judges were on the panel, only the names of the first three listed judges appear in the Compendium.  For opinions by the court sitting en banc, “Judge 1” is coded as “En Banc.”
9. Opinion 1
“Opinion 1” captures the agreement among the panel members for the prevailing outcome in a decision.  The document was coded as “Unanimous” when all members of the panel completely joined in the prevailing opinion, while “Majority” was recorded if the prevailing opinion was not unanimous.
10. Opinion 2 and Opinion 3
“Opinion 2” and “Opinion 3” provide information on additional opinions written by judges on the panel who did not fully agree with the prevailing opinion.  Common examples are “Dissenting” and “Concurring.”  There is no entry in these fields if there are no additional opinions, such as in instances when the prevailing opinion was unanimous.
11. Opinion 1 Author
“Opinion 1 Author” contains the last name of the judge who authored the prevailing opinion.  In the case of a per curiam opinion, this field is coded as “Per Curiam.”
12. Opinion 2 Author and Opinion 3 Author
The remaining opinion author fields are for the authors of additional opinions in the document, typically dissents and concurrences.
13. Notes
The “Notes” field indicates anything particularly unusual about the document and identifies the corresponding record for a duplicate.  It also describes resolutions of particular issues that arose during coding.
14. Tribunal of Origin
A recent addition to the Compendium is information about the specific tribunal from where the appeal originated.  The purpose of this field is to provide more specificity than just “DCT” or “PATO.”  Currently, the Compendium contains only data on the “Tribunal of Origin” for appeals arising from the USPTO.  This information is coded as “BPAI,” “PTAB,” or “TTAB.”
C. Data Verification
Because multiple researchers collected the documents used in the Compendium at multiple times over the span of several years, the data underwent additional verification in 2017.  Verification involved re collecting all of the information on the Federal Circuit’s website and comparing it to an export of relevant fields from the Compendium to determine whether any records were missing or duplicated.  That process revealed some minor inconsistencies with the Federal Circuit’s current website, mainly consisting of formatting issues.  A small number of new documents were added and others flagged as duplicates.  The biggest issue identified through this verification process was the discovery that about seventy decisions in appeals arising from the district courts from September 1, 2012, through April 1, 2013, were no longer on the Federal Circuit’s website.  These decisions are included in the Compendium.  A follow-up comparison of the results of a Lexis search to the Compendium for this period revealed an additional nineteen missing decisions arising from the USPTO from that time period.   A list of the specific missing decisions was sent to the court’s webmaster.
In addition, fifty records from each year—over 10% of the database—were re-collected and coded by a single research assistant and compared to the corresponding record in the database.   Due to the issue identified above, data from the period September 1, 2012, through April 1, 2013, were not included in this analysis.  This review revealed a generally high degree of agreement among coders.  However, it also revealed a handful of systematic data cleanup tasks that were necessary (for example, the “Opinion 2” field for some records had been coded with “Dissent” while others were coded as “Dissenting”). Another verification review and inter-coder agreement assessment was subsequently conducted.  Analysis of this comparison indicated extremely high agreement, particularly when two systematic coding differences were addressed.  The results of this analysis are contained in Appendix K.
Another verification step compared counts of a particular type of document—Rule 36 affirmances—to the counts obtained from a commercial database.   The results of this comparison align very closely.
Finally, note that while no collection of data is perfect, data can have greater or lesser amounts of uncertainty.  In general, the data set currently contained in the Compendium is highly objective and has been subjected to extensive verification.  That said, there is always room for improvement.  As one example, not all appeals to the Federal Circuit that arise from the district courts are conventional patent infringement cases.   A future project involves distinguishing patent cases from other types of cases arising from the district courts.  A major goal of the Compendium is continued improvement to the database while ensuring that all future use of the database is reverse-compatible with past uses in order to maximize the ability of researchers to conduct inter-study comparisons.
III. DESCRIPTIVE STATISTICS
This Part draws upon the Compendium to provide metrics on the documents released by the Federal Circuit on its website and the decisions it has issued.  The numbers used in these figures can be found in the appendices.
A. Types of Documents
The following graphs show the breakdown of types of documents in the Compendium by year.  As noted above in Part II, there are no Rule 36 affirmances in the database prior to August 2007.   This is not because there were no such affirmances—there were.   However, those decisions were not released on the court’s website.
Figure 1:  Types of Documents in Appeals Arising from the  District Courts (2004–2017)
Figure 2:  Types of Documents in Appeals Arising from the USPTO (2004–2017)
Many of the documents recorded in the Compendium are opinions and Rule 36 affirmances.  However, Figures 1 and 2 also show that between 2009 and 2014, the Federal Circuit released a large number of orders on its website.  That number declined in 2013 and dropped even more precipitously in 2015.  This is a data availability issue rather than a reflection of the actual number of orders issued by the court.  In other words, during the period 2009 to 2014, the Federal Circuit apparently decided to release its orders via its website but then ceased doing so in 2014 except in certain instances.  As of 2017, very few orders are released on the Federal Circuit’s website.   The court also releases a small number of errata each year.  These typically involve minor edits to a previously issued document.
B. Decisions by Court of Origin
This Section focuses on those documents that are the primary subject of legal scrutiny:  appellate decisions reviewing a lower tribunal’s determination.   “Decisions” in the Compendium consist of documents classified as opinions—both precedential and nonprecedential—and affirmances under Rule 36, which are necessarily nonprecedential.   Because each record in the Compendium corresponds to an individual document issued by the Federal Circuit, decisions that resolve multiple appeals that the Federal Circuit consolidated into a single proceeding are treated as a single decision.
Figure 3 shows the number of decisions arising from the district courts and USPTO.   As a reminder, the Federal Circuit hears appeals arising from other agencies as well—most significantly, the Merit Systems Protection Board and Court of Appeals for Veterans Claims, which—though not included in the data set—together comprise approximately 20% of the court’s docket.
Figure 3:  Decisions by Tribunal of Origin (Jan. 1, 2008–Dec. 31, 2017)