Denice R. Preston
It is unlikely that a glance at the top-line figures of company growth will reveal anything particularly radical. To test the drivers of performance, statistical tools can be used to understand trends that lead users to grow, retain, or engage.
There are many ways to measure the impact of growth
Let’s say, for example, that a publicly traded company wants to know how a new product has affected its valuation. To estimate the true impact, you must account for:
The market’s performance on that particular day in relation to the security.
Effect of other relevant information about the company released at the same moment.
Even in the absence of new information, it is easy to see that prices of securities and user behavior are constantly changing.
Impact on the long-term, based on a statistically significant increase in prices.
In a private firm, the same analysis could be performed on changes in active users or clients in both the short and long term, as well as their correlation to the stock price. The same applies to metrics such as retention and engagement depth.
This rounded analysis will allow companies to focus their limited resources on more reliable information and not be misled by what may appear to be a reaction from the market or users but, in reality, represents nothing more than a random fluctuation. It is worth the initial effort to create a statistical model to separate the signal from noise. The insights it provides to a business’s growth efforts can be invaluable. This is an iterative approach that can be easily updated (and, in many cases, automatically) as new data comes in.
Choose the Metric Target to Test
A company’s measurement efforts should focus on at least one dimension of growth.
TTop-line growth is defined as the change over time in total sales, active users/clients, or revenue.
Retention is the average life expectancy of a user or client.
The level of engagement between users and clients is defined by the frequency with which the core actions are taken or the volume of transactions made via the platform.
The company’s value can be conceptualized as the area formed by the triangle formed by the three points. The value of the company is severely limited if one dimension collapses. Although I agree with many investors and founders that “a few loyal users are better than many like-minded ones,” I don’t believe this contradicts top-line growth, strong engagement, and retention. The level is less important than the trajectory, and starting with a small group of dedicated users will set the best conditions for long-term success.
The company can either test different models for each or use tools such as a href=”https://en.wikipedia.org/wiki/Simultaneous_equations_model”>simultaneous equations/a> to link them more directly. The company can either use different models or tools like simultaneous equations in order to connect the metrics more directly. In my experience, marketing and PR efforts are often hampered by a lack of rigorous analysis of whether the company receives a return on investment. Some metrics are recorded almost always, like total views, shares, and clicks. But these are means to an ending. The next question is whether the company will see a return on its investment.
How to Choose the Benchmark for a Single Event
Let’s start with a simplified version of an event that happens only once. Imagine a company that releases a product update on Day 0 or publishes a major PR story and wants to know if it is a step in the right direction in terms of growth. To determine if a company has received a signal that it should continue similar efforts, it is necessary to know how much the company increased compared to how much the company would have without the event.
A regression model can be used to estimate benchmark growth. This model predicts the development, retention, or engagement of a company based on internal and external variables. In some cases, it is possible to isolate users who are affected by the product update. This allows direct A/B tests with a control. However, this is not true for large-scale business, product, or PR efforts that impact all users in a similar way. Although there are many excellent resources for this type of testing, they can be expensive for early-stage businesses.
The following variables can be taken into consideration when constructing this model:
Customer trends differ from sector trends because they focus on the growth of customers who are interested in your industry, regardless of whether they already do business with you.
The S&P500 plus sector-specific sub-indicesYou should consider if your clients are financial companies or may be affected directly or indirectly by the capital markets.
Macro variables like interest rates and exchange rates interest rates and exchange rate may have an impact on the competitiveness of your offer, depending on your business model.
Referral rates and internal drivers are examples of such drivers. The growth of any company is the result of a combination of external and internal factors. It is important to monitor internal metrics, including the referral rate of current users, which could have a significant momentum effect, user satisfaction ratings, social networking activity, etc.
Seasonality/cyclicality: You can use indicator variables that equal 1 when a condition is met (for instance, if the month falls in the holiday season) and 0 if not to control the effect of any month/day of week relevant to your users’ activity.
It is also important to consider the timeframe for each variable. Some variables are leading (the stock exchange, for example, is heavily based upon expectations), while other variables, such as user satisfaction rates, are based on past experience and may have relevance to expected growth.
In terms of regression, I would recommend starting with Ordinary Least Squares and then moving on to other functional forms if there are specific reasons. OLS allows for a more direct interpretation than other forms of analysis. Modifications to OLS include logarithmic regression for nonlinear variables and interaction variables (such as current customer satisfaction or social media activity) and square variables you think have disproportionate effects when larger. Logarithmic regressions would be a good fit since growth should be exponential.
Consider the frequency of your users’ actions or purchases when determining the appropriate time interval to measure the impact. Remember that when using timeframes greater than one day to calculate active users, the weekly total is not the daily total. If I used your product each day of the week, I would be included in a daily analysis. If you change to a week-by-week study, then I would only appear once. Therefore, adding up the days individually would be an overcount.
This model then allows you to estimate expected growth/retention/engagement for any given moment or ongoing period based on the performance of these explanatory variables. The abnormal part of the growth is the difference between the expected growth and what was actually observed after an event. Divide this abnormal growth by the standard deviation to determine the likelihood that the eccentric component occurred by chance. A result of 1,96 (being about two standard deviations from the predicted value) is usually used as a cut-off to determine that the abnormal component did not happen by chance.
When considering retention and engagement in the context of cohorts, you can either view the change between successive cohorts by holding the values constant or the overall evolution of retention and involvement over time without breaking down the cohort.
