Legal Metrics: A New Model for the 21st Century
By Roger Potter, TyMetrix
“If you can measure that of which you speak and can express it by a number, you know something of your subject; but if you cannot measure it, your knowledge is meager and unsatisfactory.” LORD KELVIN (mathematical physicist and engineer)
Legal metrics have traditionally been quite basic, consisting of one variable only or minimal composite calculations. Such simplistic legal metrics provide insight into the performance of a corporate legal department or law firm but come up short regarding how a department or firm compares to others in the market. Historically, measures such as average billing rates, profits per partner, or attorney leverage have been used to assess whether a firm or legal department was getting the best outcomes for legal matters at the most effective cost. However, these measures fail to consider other performance metrics, external factors, and benchmarks. Thus the findings often reflect what is happening to a firm rather than how the firm is performing.
Fortunately, these shortcomings can be addressed today. The availability of enhanced data along with developments in computing power and analytics enable legal professionals to more precisely define performance metrics. This article discusses how law firms and corporate law departments can use a new analytic model that evaluates the total cost of a case to determine the value a firm can provide.
Credible and Relevant Benchmarks
Benchmarking is a systematic process for identifying and implementing best or better practices, according to the Business Performance Improvement Resource. In order to do this effectively, it is critical that comparisons are being made to the right groups. Law firms and corporate legal departments cannot simply compare key performance dimensions with a select group of competitors or the legal industry in general. In legal services, there are many exogenous dimensions that drive differences in performance, such as demographics, jurisdiction, industry, matter type, availability of resources, economic factors. Many of these factors are outside of the control of the firms and legal departments but need to be controlled for if a reliable and useful comparison is to be made. Deriving the right benchmark groups involves a combination of data analysis and business acumen; it is neither a pure mathematical exercise, nor is it as simple as making selections at a very high level.
To truly understand how a firm is performing relative to its peers or for legal departments to make comparisons regarding outside counsel, it is necessary that comparisons across key performance measures can be made. It is not enough to look at billing rates, staff allocation, process or outcomes in isolation to determine the performance of a firm or decide which firms to work with. Having the lowest rates does not make a firm more competitive or cost effective if there is a disproportionate amount of work being done by partners that could be done by associates or paralegals. Nor does it make a firm a better performer if it has favorable staffing allocations, but their processes lead to longer durations for tasks or to inferior outcomes. The goal has to be to evaluate firm performance by quantifying and assessing all of these factorssimultaneously. A very good proxy for firm performance is total cost of case (TCC), which can be calculated by evaluating these factors with a multivariate modeling approach.
Controlling for Exogenous Factors and the Power to Analyze Data
Many of the factors that are built into some of the measures traditionally used to assess performance of firms, especially things like profit per partner, are impacted by factors that are not controllable by the firm. Profit is driven by the economy, the labor market, demand for legal services, the availability of sourcing options, etc. These factors cause a firm’s profit to move up and down, but are poor reflections of firm performance. While profit per partner is an important measure for the leadership of a firm to assess the financial performance of the business, for the purposes of how efficient the firm is it is a particularly poor measure, as profit can be manipulated by shedding partners just as easily as by actually performing well. This is also true when attempting to assess a corporate legal department’s efforts to most effectively and efficiently manage their legal spend across their set of matters and available firms. Exogenous factors hence need to be identified and controlled so that a fair comparison can be made across firms, and improvements can be isolated over time. If this is not done, poor assessments of performance will be made and ultimately lead to poor business decisions.
The reality of exogenous factors has always been there; in the past, though, there was little that could be done to resolve this fact. In the past 25 years or so, there have been tremendous advances in the capacity of computers, both from storage and processing perspectives. Advances in analytical software have also been critical to the development of this capability. Products like SAS, SPSS and R possess incredible computational power, providing the ability to apply state of the art analytical techniques through stored procedures so that the analyst can focus on interpretation and implementation, rather than programming the models. At the same time, vast quantities of data that satisfy the attributes that are needed to make it valuable for analytical purposes: accuracy, completeness, integrity, relevance, and timeliness, have been amassed through the use of intelligent systems, according to the International Association for Information and Data Quality. Big data, computing power, software and the emergence of the quantitative analyst have converged in the legal services industry at a time when the economic environment is driving firms to show how efficient and effective they are and clients are demanding this level of information to ensure that they are getting the best value for their money.
A New Model
A more meaningful model than one based on profits per partner or other exogenous factors is the constrained optimization model. This model identifies the optimal performance of a firm, given the constraints that external factors impose on it. In this case we want to minimize the total cost of a case (TCC), which can be defined as a combination of expense and outcome. Theoretically, additional dollars invested in a case will lead to an improvement in the financial outcome. The fundamental assumptions are that investing zero dollars will lead to the worst possible financial outcome, that an incremental investment in legal services will initially reduce the financial outcome by an amount greater than the investment, and that eventually it will cost more to add legal services than the reduction in the financial outcome. Therefore, the firm will minimize its TCC when the marginal cost of additional legal support is equal to the marginal improvement in the financial outcome.
Expense is a function of process, rates, and role utilization; these will be explained in greater detail below. In other words, the expense will be driven by what is done, who does it, and how much they charge. It is very likely that what is done and who does it will impact the outcome as well, which is an interesting area for analysis, but beyond the scope of the current exercise.
There are constraints that will drive differences in all of our independent factors that should be controlled for whenever possible. For example, geographical differences in the availability of experienced paralegals can have a significant impact on the amount that this role can be utilized and ostensibly on the overall cost. Geography will also have an impact on rates, as will practice area, firm size (which in the short run should be considered exogenous), and general economic factors. These constraints need to be considered and wherever possible, built into the model.
The mathematical form of the model is:
Where: TCC = ƒ(expense + outcome),
Expense = ƒ(process, rates, role utilization);
Subject to: economic factors, geography, practice area, industry served, etc.
- Process: This is how legal professionals choose what to do in order to complete a matter. In its simplest form, it is a mixture of the tasks that are chosen and the amount of time spent on each of the tasks.
- Rates: Often rate comparisons are used to compare firms or other providers, but in this context, internal rates across roles and timekeepers are as important as the rates relative to competitors.
- Role utilization:This is the final piece of the expense portion of the model. The role impacts both the rates and amount of work. It is often argued that a partner can do certain tasks significantly faster than associates or paralegals, and while this is likely to be true in many—if not most—cases, this needs to be balanced by the respective cost differentials. If a partner can do something in half the time of a paralegal, but costs three times as much, there is not a benefit to having the partner do the work, unless there is a quality component that can be measured.2
- Outcome:The outcome of the matter needs to be included in the assessment of the performance of the firm. This can be a mathematical exercise, if there is some kind of payout as a result of the case or it can be qualitative in nature. To the extent that there is a financial exchange at the end of the matter, a very good approach is to use the difference between the observed and expected outcome. It is possible to forecast an expected result and compare to that, rather than assessing an outcome based on what happens. This way the valuation is based on what the individual or firm adds to (or detracts from) what an average attorney would do. This is valuable because it measures contribution, rather than absolute value. Also, there has been much work done in assigning values as part of a qualitative assessment process. This can be used as a proxy for the value of the outcome and can be sized based on the magnitude of the matter. This is somewhat more tenuous than pure financial data, but in many situations, there simply isn’t financial data to be had.
These exogenous factors need to be used as constraints when doing modeling. They are important from a benchmarking perspective, and should also be used as controlling factors within the models that are developed to calculate TCC. Economic factors, especially labor market related measures, will have a significant impact on rates and availability of resources, so it is important to take this into account when assessing a firm based on its utilization of staff, rates and processes. Jurisdictional factors should be taken into account when assessing outcomes, the tort laws of a given state will have a huge impact on the outcomes of certain types of cases and should therefore be controlled for in the modeling process. Finally, the types of work done and the clients served will impact a lot of the controllable variables, so these need to be contemplated as well when building out the models and comparison framework.
How to Use the Model
The TCC model can be used by both corporate legal departments and by law firms, although each may utilize different data and have different purposes. In both cases the user will be able to evaluate the overall performance of a firm based on process, role utilization, rates and outcomes. More importantly, they will be able to drill in to the data and see what is happening at a more granular level. This ability to slice and dice is the true power of the TCC model, because it enables the user to understand why things are happening as well as what is happening.
Corporate legal departments will be able them to negotiate more effectively with their outside counsel across all of the key dimensions and not just on timekeeper rates. They will also be able to use the tool for selection of the most effective firm for specific cases or types of matters in particular locations. With the use of industry data, they can also benchmark the support that they are getting against others, especially those with similar business and legal issues.
Law firms will also be able to benchmark themselves against their peers. They will be able to see where they are competitive and where they are potentially leaving money on the table. There is also the ability to assess leaders within the organization as to their strategy, resource utilization and outcomes. It can also be extremely useful in the calculation of what to charge in alternative fee arrangements, as well as provide data to support different support models as this model becomes more prevalent.
Companies that take advantage of advanced analytics of law firm performance will be rewarded with both lower expenses and improved outcomes. Forward looking firms can also utilize advanced analytics to drive a best practice culture into their business and show prospective customers that they are the best choice based on numbers other than rates. Companies and firms that take advantage of these capabilities first will win in the market; they will have superior information to manage their businesses and make data-based decisions, giving them an edge over their competition.
Roger Potter is the Director of Analytics at TyMetrix. Potter brings nearly 20 years of experience to TyMetrix. He is responsible for Legal Analytics, encompassing market analysis, predictive modeling, and custom analytics.
This document and any discussions set forth herein are for informational purposes only, and should not be construed as legal advice, which has to be addressed to particular facts and circumstances involved in any given situation. Review or use of the document and any discussions does not create an attorney-client relationship with the author or publisher. To the extent that this document may contain suggested provisions, they will require modification to suit a particular transaction, jurisdiction or situation. Please consult with an attorney with the appropriate level of experience if you have any questions. Any tax information contained in the document or discussions is not intended to be used, and cannot be used, for purposes of avoiding penalties imposed under the United States Internal Revenue Code. Any opinions expressed are those of the author. Bloomberg Finance L.P. and its affiliated entities do not take responsibility for the content in this document or discussions and do not make any representation or warranty as to their completeness or accuracy.
© 2012 Bloomberg Finance L.P. All rights reserved. Bloomberg Law Reports ® is a registered trademark and service mark of Bloomberg Finance L.P.