Hi I’m Valerio, software engineer and creator of Inspector.

As product owner I know that being able to prevent users from noticing an application issue is probably the best way for developers to contribute to the success of a software-based business.

We could talk about user complaints, customer churn, a thousand other things, but in short, in a highly competitive market any application error can expose developers to competitive or even financial risks. We publish new code changes almost every day, and it’s quite impossible to anticipate all the problems that could happen after every release.

It’s too important for developers to catch errors on their products —before— their users stumble into the problem drastically reducing negative impact on their experience.

I’m refine and search every day new metrics to move my business forward, and my product itself is a tool that provide instant and actionable metrics to its users, so I study and practice a lot to find the best possible information to avoid unnecessary risks.

I’m not interested to create charts that looks good (even if they are), my priority are useful, indeed needful metrics to distinguish between something that doesn’t need to be rushed and something that needs immediate attention to keep my application (and my business) stable and secure.

Why averages don’t work?

Anyone that has ever made a decision uses or has used averages. They are simple to understand and calculate.

But although all of us use them, we tend to ignore just how wrong the picture that averages paint of the world is. Let me give you a real-world example. 

Imagine being a Formula 1 driver.

Your average “execution” time for a lap is comparable with the top three in the ranking, but you are in fifth position. 

According to the average, everything is fine. According to your fans, it’s not so good.

Your “Team Principal” – the person who owns and is in charge of your team during the race weekend –  knows that relying on averages is not a good way to understand what’s going wrong. He know that, when it comes to making decisions, the average sucks.  When calculating the average, it’s likely that in some races you’re so fast that you can make up for the next four races with bad performances.

As an F1 driver you can compare your “execution” time and results with other drivers, but with your application you are alone, the only feedback you have is customer churn.

Your team principal knows that focusing too hard on the best performances is not so useful to understand what’s going wrong and how to fix it (car settings, pit stop, physical training, etc.).

He recalculates the average taking into consideration only the worst 5% of your races (95th percentile). Isolating these executions from the noise he can now analyze them and clearly see that every time something goes wrong it is because of the pit stop.

Measuring in real time the worst 5% of your application cycles gives you the same opportunity. You’re able to understand what is going wrong when your application slow down (a too time-consuming query, slow external services, etc.) and avoid bad customer experiences, because you always have the right information before your users stumble into the problem.

In a typical web back-end we experience the same scenario: some transactions are very fast, but the bulk are normal. The main reason for this scenario is failed transactions, more specifically transactions that failed fast, not for bugs but due to user errors or data validation errors.

These failed transactions are often magnitudes faster than the real ones because the application barely starts running and then stops immediately; consequently, they distort the average.

The secret to using averages successfully is: “Measure the worst side”

Inspector shows you the “execution time analysis” of the worst 50% (Median) and the worst 5% (95th percentile) of application cycles.

As you can see the median (blue line) is rather stable but has a couple of jumps. These jumps represent real performance degradation for the majority (50%) of the transactions. The 95th percentile (red line) is more volatile, which means that the outliers slowness depends on data, user behavior, or external services performance.

In this way you will automatically focus only on transactions that have bad performance or problems that need to be solved. 

Inspector eliminates any misunderstanding and offers a dashboard that informs you directly about things that can cause problems to your users and even to your business, including errors and unexpected exceptions, as you can read about in the first part of this series [Laravel Real-Time monitoring & alerting using Inspector].

Automatic alerting

In real-world environments, performance gets attention when it is poor and has a negative impact on the business and users. But how can we identify performance issues quickly to prevent negative effects? 

We cannot send out alerts for every slow transaction. In addition, most operations teams have to maintain a large number of applications and are not familiar with all of them, so manually setting thresholds can be inaccurate, time-consuming and leave a huge margin for errors.

1 — Blue line still flat, Red line jump (low priority)

If the 5% degrade from 1 second to 2 seconds while the 50% is stable at 700ms. This means that your application as a whole is stable, but a few outliers have worsened. It’s nothing to worry about immediately but thanks to inspector you can drill down into these transactions to inspect what happened.

Inspector metrics don’t miss any important performance degradation, but in this case we don’t alert you, because the issue involves only a small part of your transactions and is probably only a temporary problem! Thanks to Inspector you can check if the problem repeats itself and eventually investigate why.

2 — Blue line jump, Red line still flat (high priority)

If the worst 50% moves from 500ms to 800ms I know that the majority of my transactions are suffering an important performance degradation. It’s probably necessary to react to that.

In many cases, we see that the red line does not change at all in such a scenario. This means the slow transactions didn’t get any slower; only the normal ones did with a high impact on your users.

In this scenario Inspector will alert you immediately.

Conclusion

Your team can now work for a better pit stop and you will soon be able to compete with the best drivers in the league. Measure continuously potential problems is the secret behind the great Formula 1 teams to achieve success not once, but to remain in the top teams for all the years to come.

Inspector is a developer tool that drastically reduce the impact of an application issue because you will be aware of it before your users stumble into the problem.

Thank you so much for reading it. To make Inspector more sophisticated and mature, it will not be possible to accomplish without your help. Don’t hesitate to share your thoughts on the comment below or drop in live chat on our website! Let’s make it together.

By LKB

As previously reported, I had a ringside seat for yesterday’s SCOTUS arguments, and Dan posted some of my quick observations from the argument. Now that I have had a bit more time to digest things (as well as some food and sleep), I want to go into a bit more detail on how I perceived how the argument went down.

Mootness

As has been widely reported, most of the time spent during oral argument involved addressing whether New York City’s last-minute maneuver (i.e., after the Supreme Court granted cert, the City and the State changed the law in a transparent and brazen attempt to evade appellate review) rendered the case at bar moot.

Mootness is a legal doctrine that provides that when there is no longer any actual dispute between the parties for the court to decide, an appellate court should dismiss the case rather than enter advisory or hypothetical opinions. However, it is a complicated area of law that has many exceptions, many of which were discussed yesterday.

Courtesy LKB

Contrary to how the media (and unfortunately some Chicken Littles in the firearms commentariat) have tried to spin the amount of time at argument that was spent discussing mootness, that fact doesn’t really mean anything. In appellate advocacy, you go into oral argument with an outline of points you would like to make…usually a very small subset of the arguments made in your briefs.

However, you are not in control of things, and your argument goes where the most active questioners on the Court want to take it. This is especially true at the Supreme Court, where the van Moltke maxim that “no battle plan survives first contact” is typically the case.

In NYSR&PA, the anti-2A wing of the Court (Ginsburg, Breyer, Kagen and Sotomayor) are all known as active questioners at oral argument, as opposed to Thomas, who almost never asks anything at all, and Roberts and Kavanaugh, who tend to ask very few.

As a result, the liberal wing of the Court dominated the time at oral argument with their questioning. However, just because they asked the most questions (and thus made the attorneys spend most of their argument answering them), that simply does not signify anything.

LKB for TTAG

So, how will the Court decide the mootness issue? As I expected, Ginsburg and Sotomayor were more than willing to swallow NYC’s brazen tactical maneuver. While I thought that Kagen and Breyer might have some shred of integrity on this procedural issue (blessing this kind of post-cert gamesmanship would set a horrid precedent for all sorts of cases – especially many that those two justices typically care about), that possibility evaporated at oral argument. Both of those justices were clearly on board with letting the City get away with it in order to avoid a potential expansion of Heller.

On the other hand, both Alito and Gorsuch did not mince words about how they felt about NYC’s chicanery. Gorsuch acidly remarked on NYC’s “Herculean efforts” to evade Supreme Court review after cert was granted. Thomas and Kavanaugh asked no questions, but their positions in past cases leave me with little doubt that they are not going to vote to hold the case moot.

That leaves the deciding vote on the mootness issue with Roberts. Early media reports were crowing that Roberts’ questions indicated that he was leaning toward finding the case moot. From what I saw (and confirmed in the transcript), I simply did not see that.

The Chief Justice asked only two questions, one of which was a bit of a trap that adduced a concession from NYC’s counsel that dismissing the case as moot would prejudice the plaintiffs’ ability to seek damages for violations of their rights.  If anything, I think Roberts’ questions cut the other way on which way he may be leaning.

Courtesy Kevin Hulbert

Additionally, if Roberts was going to wimp out on the mootness issue, I suspect he would have already done so. The Court had several earlier opportunities to dismiss the case as moot, and if he was so inclined, Roberts could have voted with the liberals and done so. He did not.

Of course, as the Obamacare decision illustrated, Roberts can certainly be a squish, and thus he could well sell us out on this issue. However, nothing I saw at oral argument supported the spin the media put on it.

Indeed, I suspect the media is pushing the “it’s going to be dismissed as moot” narrative for its own purposes (e.g., to be able to decry a Roberts “change of position” on mootness as evidence that the court is broken and needs to be packed, etc.)

LKB prediction: 5-4 denial of the motion to dismiss the case as moot. However, if we see a cert grant in the next few weeks on another 2A case — there are several being held due to the grant of cert in NYSR&PA — then all bets are off.

The Merits of the Case

Because of the near-monopolization of the oral argument by the liberal wing of the Court on the mootness issue, very little time was spent on the merits (absolutely none on the Commerce Clause or Right to Travel issues). However, there were a few interesting moments.

The highlight of the arguments on the merits was a spectacular trap that Justice Alito sprang on the attorney for the City. Questioning him on NYC’s change of its laws, he asked, “Are people in New York less safe now as a result of the new city and state laws than they were before?”

Clearly surprised at this seemingly out-of-the-blue question, counsel responded that they were not less safe. Alito then pressed him to concede that there was thus no actual basis for the City to claim that the transportation ban was essential to public safety.

Counsel attempted to tap dance away from that, claiming the restriction accorded with the history of acceptable regulations under the Second Amendment.

Alito then pressed counsel on whether a total ban on transportation by premises license holders could possibly be constitutional. The City’s position – and Second Circuit law – “cabin” Heller to possession of a weapon in the home.

I expected him to respond “yes” and then just take the heat. Instead, he admitted that such a complete ban would violate the Second Amendment.

Justice Alito then pounced: “If that’s what it means, you’re conceding that the Second Amendment protects the possession of a firearm outside the home under at least some circumstances?” Counsel again tried to tap dance away, but again conceded that was a “fair way to look at it.“

So much for the City’s argument that Heller applies only to possession inside the home.

Counsel for both NYSR&PA and the Solicitor General pushed application of the “Text, History, and Tradition” test as the applicable standard, rather than strict scrutiny. In what I took as a transparent shot at Justice Kavanaugh, Justice Sotomayor remarked that she viewed the “Text, History, and Tradition” test as a “made-up standard.”

Courtesy Matt Laur

Needless to say, the fact that standards for decision – or even new constitutional rights – might have been “made up” by judges has hardly concerned Sotomayor in the past.

At another point, she remarked that questions about whether certain types of weapons were covered by the Second Amendment was not before the Court yet. I might be reading too much into it, but I took her comment and the way she asked it as a recognition that regardless of what happens in this case, she knows that the votes are there to take other 2A cases.

Near the end of the argument, Justice Ginsburg (who looked very frail, but nevertheless was engaged and asked a number of probative questions in both of Monday’s arguments) asked the City’s counsel whether, because the transportation ban forbade taking a licensed gun to a second house (whether in or out of the city), that would require a license holder who wished to be armed at home to acquire two guns — one for each house — and leave one gun at an unoccupied location at all times, which she seemed to intimate would be less safe than transporting one gun between them.

To me, she plainly was teasing an argument that perhaps the NYC transportation ban could be struck based on intermediate scrutiny, perhaps to try and tempt Roberts into reversing on narrower grounds. (Query why she would do this if she thinks Roberts might squish on mootness.)

At one point in the argument, Justice Breyer made a comment that indicated that he still does not accept that Heller was correctly decided…which caused his neighbor, Justice Thomas, to lean over and engage him in a whispered conversation.

My read on the merits: nothing has changed, and things are as they have been. There is a wing of the Court (Ginsburg, Breyer, Kagen, Sotomayor) that is adamantly anti-2A and would gladly reverse Heller if they had the chance. There are four votes (Thomas, Alito, Gorsuch, and Kavanaugh) who go the other way, and would likely expand Heller significantly.

Roberts is also at least nominally in this camp, but the question remains whether he will succumb to the Beltway social, media, and political pressure as he did in the Obamacare decision.

My prediction: if they reach the merits, 5-4 to reverse.  Smart money would be that Roberts will write the opinion, but I’ll go out on a limb and predict that he’ll assign it to Kavanaugh, and will adopt the “Text, History, and Tradition” test for 2A cases. Concurrence by Thomas, Alito, and Gorsuch, pushing for strict scrutiny and an end to lower court gamesmanship on 2A cases.

We’ll see . . . .