Netflix’s Production Technology = Voltron

by Chris Goss

Change management is hard. In everyday production, there are numerous factors working against embracing change. Limited preparation time, whole new show = whole new crew, innumerable planning variables, and the challenge of driving an operational plan based on creative instincts. These are problems that technology is not yet built to solve. Time, training, and education can and will make a dent in our efforts, but creative planning is nuanced, and by nature, human.

So, where do we start? What can production management technology do now to pave the way for future change? Having spent the past two years building Prodicle, our production suite of apps, we hit several pockets of success, while learning from numerous obstacles. Others have ventured down this path as well, and there are several start-ups and moderate size companies that reach out to us about their product offerings. It’s exciting to see others with the same level of passion and enthusiasm we have for improving the way entertainment gets made. After many meetings, memos, pitches and 30-day trials, there are a number of trends we often find ourselves discussing with all production SaaS providers and we think it’s worthwhile to share these key development pillars to start an ongoing conversation about how we’re thinking of pursuing these problems. Some have manifested themselves in how we’ve built Prodicle, others are ideas and pipedreams we hope to pursue in time. If you’re a SaaS provider seeking to dig your heels into the entertainment production space, here are some common themes we often discuss with our partners, our crews and our software development teams.

Modularity is key.

We built Prodicle with the intention of being modular, which is critical based on the wide variety and volume of content we produce. We are not a domestic TV studio. We are a global streaming studio, which means we need to produce entertainment that satisfies the different tastes of our members all over the world. Live action films, multi-camera comedies, weekly talk shows, global competitions, international crime dramas, documentaries about the Rajneesh — all forms of content for all audiences. This means that one giant monolithic software solution will not work — build, buy or otherwise. We are often pitched the dreamstate of software: one end-to-end solution to rule them all. While all-in ecosystems may work for smaller studios/production companies, the variables across our content slate makes it very difficult to successfully operationalize. As such, our offering will always be a modular ecosystem of connected solutions, some things built by us, other things bought from vendors, and a whole bunch of stuff in between. The beauty is twofold — products that work great on their own, products that work better together. It’s like Voltron. Season 7 now streaming.

It’s perfectly fine for a singular solution to solve one problem really well — in fact, this is often preferred. If your solution seeks to do seven things, chances are we’ve already solved for five of them — we just need the remaining two. Those two become the attractive secret sauce, and if they play well in a modular sandbox, we want to hear more! But when they don’t, and we have to go all-in on multiple ecosystems that overlap in functionality, the pain of deployment and support makes it a near non-starter. Passport is Prodicle’s menu-based portal that brings multiple solutions together in one place. It’s where we seek to grow our product offering for our crews. One modular location, many products.

Identity is complex.

It’s important to have safeguards in place to protect our content. Filmmaking is collaborative and lots of people touch the final product. Safely getting in and out of our ecosystem is important to get right. If too complex, it becomes a barrier to change management and introduces less-than-ideal workarounds. If too liberal, then we face the possibility of content leaking early. It’s an exercise in balance. Where do we start? From the product/user perspective, we need one login. We cannot have a multitude of solutions all utilizing different usernames and passwords. Retail has got this figured out, the freelance work environment needs to solve it as well. If your product doesn’t offer SSO (single sign-on) it becomes an immediate point of resistance. SSO is crucial to making security as transparent to the user as possible. Second? Roles and permissions. Our Content Engineering team has built an internal solution to manage app-to-app roles and permissions in a single administrative portal. This means we can provision users across multiple apps in one location. We’d love the ability to call your APIs and set application roles individually and in-bulk from one place. It’s a user administrator’s dream. Working with tens of thousands of freelancers around the world mandates this as a requirement. SSO and APIs for roles/permissions is a biggie.

Collaboration is key.

Netflix adopted Google Cloud’s G Suite in 2013. This means our entire corporate file sharing platform is Google Drive. Thousands of users, millions of files within the cloud — accessible everywhere. When we started the studio, we were faced with a choice: a) deviate and use a different file sharing platform, perhaps different platforms for different types of content, or b) use G Suite on our productions so corporate, studio and production are all in one shared space. This was a tough call, as the variables within our content, as noted above, are extensive. However, we felt G Suite was and is flexible enough while subscribing to our core principles of modularity. Utilizing their service within our ecosystem was the fastest way to stand-up a global file sharing platform. We have hundreds of productions using the tools within G Suite, collaborating with our studio users. Instead of trying to build our own file platform, or buying a multitude of third party platforms, we started building production-centric features on top of G Suite. For dynamic watermarking, password protection and expiring links, we built Prodicle Distribution. It sits on top of Google Drive, taking advantage of the multitude of G Suite functions that would be silly for us to build ourselves, e.g. collaboration, commenting, file previews, etc. In doing this, utilizing consumer facing tech from companies whose full-time focus is workplace solutions (our main focus is great content), we’re able to leverage what works well and add what we need to that. We don’t expect consumer tech companies to build production functionality, but if we can leverage what they do well and we build what we need for us to do well (produce great content) it gives us a significant head start. We look for this in third-party production tools all the time. Again, we want production components that don’t exist anywhere else, not another folder-based file repository to store our scripts.

Data as operational currency.

There are groups, committees, and conferences where universal data IDs are endlessly discussed. While not impossible, the effort needed for unification is tremendous. Data reconciliation will be an ongoing challenge especially when data accessibility isn’t solved. As such, availability of production and studio generated data is a must-have. The production-to-studio relationship requires the real time transfer of assets and data. When data is locked behind a proprietary file format or within data architecture with no accessible APIs, the constraints significantly reduce the data’s usefulness. Studios are hampered by the time it takes for software companies to develop their UIs. This significantly limits the software’s functionality and reduces its value at the studio level. Investing in software is a dual focus: accessibility of the data and assets generated by the software and the operational efficiencies provided by the use of the software. Both needs have to be serviced as production and studio each have different data and operational needs. However, this raises a subsequent problem of…

Serving two masters.

Most third-party production software offerings service singular needs. The products are either loved by productions and disliked by studios, or loved by studios and disliked by productions. This is a real challenge because the benefit is in the marriage of the two. But as a SaaS provider, how do you prioritize user needs? By listening to the producer, the studio exec, the production assistant? We seek to answer this question, but it’s not easy practice. Most of the solutions lie within the above — separating the data from the UX. A production UX has to be attractive to a production user — someone singularly focused on executing his/her job for one production. A studio UX is macro, servicing the needs of a multitude of productions each with their own unique variances. Does this equate to doubling the investment? Perhaps from a UX perspective, but less so from a data perspective — that is, if the data is easily accessible. If a studio can centralize pertinent micro-level and macro-level data, the UX can be agnostic — built by us or built elsewhere. This is where third-party offerings can be very attractive: solutions that offer unique operational advantages in a killer UX with data and assets that are easily accessible at the studio level for macro-level needs.

Change is bigger than Netflix.

Our entire industry is on the precipice of embracing new technologies all in pursuit of providing the best tools to our productions with the goal of empowering them to focus on what matters most — creating great stories. There are significant opportunities to partner on solving operational challenges, and there are a lot of them. We seek to utilize modern technology to solve these problems by building support for a Netflix platform that can be used on any Netflix production. That platform is modular — one to utilize an agnostic offering of SaaS products, built by us, built by others. Ideally, industry-wide discussions on normalizing the data models would be ideal. Perhaps, one stop at a time. We work to solve the gaps that no studio has yet to solve, giving our creators the advantage of cutting-edge production management software. Through this line of thinking, we want to leverage what exists to move fast in solving areas that do not yet exist. But it’s a paradigm shift — both in how the collective industry builds and how our users adopt. We are still in our infancy, having barely scratched the surface and there’s still so much to be done. It’s an exciting time to be one-part entertainment and one-part tech. For more information on this initiative, check out the Netflix media blog here.


Netflix’s Production Technology = Voltron was originally published in Netflix TechBlog on Medium, where people are continuing the conversation by highlighting and responding to this story.

via Planet MySQL
Netflix’s Production Technology = Voltron

The Laravel Sell


Presumably, if you’re watching this series, you’ve already made the decision to embrace all that Laravel has to offer. However, if you’re still on the fence, give me just a moment to sell you on why I believe Laravel is the best framework choice in the PHP world.


via Laracasts
The Laravel Sell

Savage Unveils New Rascal Micro Rimfires


Savage Rascal Target XP

Savage Rascal Target XP (Image: Savage Arms)

Savage Arms announced yesterday four new versions of their Rascal, an undersized single-shot bolt-action rimfire rifle designed to offer high performance in a compact format that should be perfect for youths and smaller folks. The new versions feature synthetic stocks (and there’s a pink version of the Rascal FV-SR) and heavy, suppressor-ready threaded barrels. AccuTrigger comes standard, of course.

From the press release:

Savage is pleased to take micro rimfire rifle performance to new heights with the new Rascal FV-SR (Suppressor Ready), Rascal Target and Rascal Target XP. Shipments of these firearms have been delivered to dealers.

The Rascal FV-SR enhances accuracy with a heavy barrel featuring a threaded (1/2-28) muzzle. Like the original Rascal, the single-shot rifle cocks by lifting the bolt and unloads easily—without pulling the trigger. The design also builds superior shooting skills thanks to full-size rifle features, including a peep sight, tough synthetic stock and Savage’s user-adjustable Accu-Trigger system. The two new suppressor-ready Rascal models are available in pink or black.

The new Rascal Target is built for accuracy and adaptability, thanks to a precision hardwood stock, heavy 16 1/8-inch barrel, one-piece Picatinny rail and threaded (1/2-28) muzzle. Like the Rascal FV-SR, the Rascal Target offers full-size rifle features, including Savage’s user-adjustable AccuTrigger system and suppressor-ready barrels. Collectively, the platform offers far better safety, functionality and accuracy than competing designs.

The Rascal Target XP offers all the Rascal Target amenities in a ready-to-shoot package that comes with a mounted and boresighted 4x32mm scope, as well as a bipod sling swivel stud mount. A left-hand model of this configuration is also available.

Rascal Target, Target XP Features & Benefits

  • Precision hardwood stock
  • 16 1/8-inch heavy barrel with threaded muzzle (Suppressor Ready)
  • One-piece Picatinny rail
  • Reliable, single-shot action
  • Manual safety
  • Unload without pulling the trigger
  • Adjustable AccuTrigger
  • Feed ramp
  • Includes ear plugs and firearms lock
  • 11-degree target crown
  • 4x32mm scope (Rascal Target XP)
  • Bipod sling swivel stud mount (Rascal Target XP)

Part No. / Description / MSRP

  • 13834/Rascal FV-SR Black, 22 S, L, LR, 16.125-inch barrel / $219
  • 13835/Rascal FV-SR Pink, 22 S, L, LR, 16.125-inch barrel / $219
  • 13823/Rascal Target, 22 S, L, LR, 16.125-inch barrel / $314
  • 13824/Rascal Target XP, 22 S, L, LR, 16.125-inch barrel / $399
  • 13836/Rascal Target XP LH, 22 S, L, LR, 16.125-inch barrel / $399

(end press release)

Visit their website to check them out:

Savage Rascal FV-SR

Savage Rascal FV-SR (Image: Savage Arms)

Savage Rascal FV-SR

Savage Rascal Target

Savage Rascal Target (Image: Savage Arms)

Rascal Target

Savage Rascal Target XP

Savage Rascal Target XP (Image: Savage Arms)

Rascal Target XP

Savage Rascal Target XP Left Hand

Savage Rascal Target XP Left Hand (Image: Savage Arms)

Rascal Target XP Left Hand


via All Outdoor
Savage Unveils New Rascal Micro Rimfires

‘Calling Facebook a toilet is a little unfair to toilets’: HBO’s John Oliver flush with anger for social giant


“Last Week Tonight with John Oliver” got in its digs at tech companies on Sunday night. The show’s jab at Zillow was child’s play compared to how the host took down Facebook and the role the social giant is playing in cultures and communities overseas.

In his nearly 20-minute main story on his weekly HBO show, Oliver focused on Facebook’s impact in Myanmar in particular where internet use — and Facebook use especially — has exploded in recent years.

But the story goes beyond distracted teens and delves into serious concerns about ethnic cleansing. Certainly Facebook was not to blame for genocide in Myanmar, Olivers stressed, but the service did inflame pre-existing tensions he said. Oliver pointed to a report by independent U.N. investigators who said “Facebook has been a useful instrument for those seeking to spread hate.”

Because Oliver is a comedian, it’s not all super heavy and he does pepper his monologue with plenty of zingers:

  • “Facebook: The worst place to wish happy birthday to a friend, other than a funeral.”
  • Mark Zuckerberg, while explaining Facebook’s early motto “Move fast and break things,” said, “Sometimes we go too fast and we mess up a bunch of stuff and we have to fix it. And that’s cool.” Oliver asked, “Is that cool? That seems like the most reckless corporate motto since the Hindenberg company’s “Flay fast and smoke cigarettes in a big bag of hydrogen!”
  • “Calling Facebook a toilet is a little unfair to toilets, because they make s*it go away, whereas Facebook retains s*it, disseminates s*it to your acquaintances, and reminds you of s*it from seven years ago, all while allowing corporations to put there s*it in front of you. There is a purity and integrity to toilets that Facebook seriously lacks.”

But Oliver gets at the seriousness of the issue in Myanmar and the tensions between different ethnic factions quite convincingly, and he wonders why so much hate speech in the country stayed up for so long on the site — which has rules against that sort of thing.

“No one should be judged by the worst things people say about them on Facebook,” Oliver argued.

Watch the full video above.


via GeekWire
‘Calling Facebook a toilet is a little unfair to toilets’: HBO’s John Oliver flush with anger for social giant

The Best Hand Mixer


Photo: Michael Hession

The Breville Handy Mix Scraper was the most versatile, effective mixer of any we tried, with the most functional attachments and useful features that make it a joy to use in the kitchen. With nine speeds that can toggle from powerful to gentle, it made the best cookies, sponge cake, and pizza dough in our tests. It had the most unique and intelligent design of any mixer we tried, with a built-in timer, a button to pause mixing, a light that shines into the bowl, and rubber-coated scrapers. As a bonus, it was also the simplest to clean and store compared with other mixers. We think this superior mixer is worth its relatively high cost, but the Cuisinart Power Advantage Plus 9-Speed Hand Mixer is a steadfast budget option if you want a pared-down machine for occasional tasks.

The Breville excelled in all of our tests. It was powerful enough to blend the thick cookie dough without straining, and its coated beaters helped keep all of the ingredients from climbing up the sides of the bowl. Each baked cookie was well-mixed with an even distribution of the good, chunky morsels we wanted in every bite—chewy raisins, chopped walnuts, chocolate chips, and flakes of coconut. The sponge cake it produced had an even crumb, a delicate texture, and the least air bubbles of any we baked. It was also one of the loftiest, unlike the VonShef’s cake, which sunk in the middle and at the sides. The Breville’s dough hook was even able to mix pizza dough pretty well, whacking it around to create a single ball, whereas the VonShef and the KitchenAid created stringier doughs that we had to take out of the bowl and shape on the counter. After whipping eggs and sugar for the requisite seven minutes, the Handy Mix produced just under a quart of seven-minute frosting—slightly less than the Cuisinart— but it was still fluffy, spreadable, and delicious.

Compared with those of other mixers, the Breville’s attachments stood out for their more thoughtful design and effective mixing capabilities. Rubber-coated beaters on the Breville do a better job of scraping the sides and bottom of the bowl compared with other models’ beaters, and help with noise control. “It doesn’t make a racket if you’re mixing stuff in a metal bowl,” said Lopez, who also owns this mixer. And we didn’t have to scrape the bowl with a spatula as often as with the Cuisinart when mixing our cookie dough with the beater attachment. The Breville’s two whisk attachments actually work to whip eggs for sponge cake or meringue, while we had to use the beater attachment to get similar results with all of the other mixers, which had only one whisk. In our genoise test, we found that the Breville’s whisk attachments actually worked better than the beaters at aeration (we tried both), making a more evenly baked cake.

With its clever extra features, the Breville was by far the easiest and most enjoyable mixer to use of any we tested. A screen with a built-in timer helps keep track for time-sensitive recipes, which I found especially useful when making seven-minute frosting. The timer also has a pause button, so you can stop mixing, add ingredients or scrape down your bowl, and then resume immediately without having to speed up again. You can seamlessly transition between nine speeds with a scroll wheel, similar to one an old-school computer mouse. Even though there’s nothing wrong with pressing a button, the scrolling felt easy and familiar (and made me nostalgic for pre-Apple days). The mixer also has a light at the bottom of the machine that shines into the bowl, which I used to check if the sponge cake batter had turned the pale, glossy yellow I was looking for. None of the other mixers we tested had these features, and though they’re definitely a bonus, I didn’t think they were a gimmick—they made the baking process easier overall.

  • handmixer-lowres-1099

    An LCD panel shows a timer and your current speed. Photo: Michael Hession

  • handmixer-lowres-1103

    A light at the bottom of the Breville shines into the bowl so that you can easily see the state of your batter or dough. Photo: Michael Hession

  • handmixer-lowres-1106

    A rubber coating on the Breville’s beaters prevents them from clanging loudly against a bowl during mixing. Photo: Michael Hession

Breville says that this mixer is touch-sensitive, using the company’s proprietary Beater IQ technology to adjust mixing speeds depending on which attachments you use. We didn’t notice this kicking in during testing, but the Handy Mix was sufficiently powerful to handle all of the doughs and batters we made. It seamlessly transitioned from slow to high speeds without splashing ingredients around.

  • handmixer-lowres-1107

    Dough hooks, whisks, and standard beater attachments come with the Breville; the rubber-coated beaters help scrape the mixing bowl and are quieter than metal beaters, the two whisks aerate batters more effectively than mixers with only one whisk, and the dough hooks can knead dense pizza dough. Photo: Michael Hession

  • handmixer-lowres-1111

    All of the attachments store neatly in a clear plastic case, so you won’t lose them to the depths of a drawer. Photo: Michael Hession

This mixer was the easiest to clean and store, too. Instead of pressing a button to eject beaters, you can pull a loop to release them, which requires much less effort than with other mixers. This is a small detail, but it makes using the Breville that much easier—in contrast, I had to put my full body weight on the VonShef’s eject button to try to get the beaters out. The Handy Mix has Breville’s patented Assist Plug (found on most Breville appliances), which has a loop for your finger so that you can pull the plug out of the wall more easily. I found myself wishing that more of my kitchen appliances and home electronics had this feature.

A loop on the Breville’s plug makes it easy to pull from the socket; it’s just one example of the mixer’s thoughtful design. Photo: Sarah Kobos

All of the beaters are dishwasher safe, and you can easily wipe down the mixer with a damp cloth (although all hand mixers have vents—to keep the motor from overheating—that can trap food). A plastic case slides cleanly onto the base of Breville for storage, whereas you have to align the Cuisinart’s storage case with the mixer’s vents to attach one to the other. The Breville’s case neatly holds all of the beater attachments and the cord. With some other mixers, you either have to save the box the machine comes in, or store components in a bag. We like that the all-in-one system keeps the mixer’s small parts together, so that you don’t have to fumble around in a drawer for a single missing dough hook.

Epicurious also likes this mixer, and Michelle Lopez told us she uses it for many baking projects. “It’s quiet, powerful, and comes with a bunch of bells and whistles that make it great for precise baking,” she said. The Breville has a one-year limited warranty. We’ll continue to test it over time to see how it holds up.


via Wirecutter: Reviews for the Real World
The Best Hand Mixer

Bamboo Skewer Chopping Board

Bamboo Skewer Chopping Board

Link

Pask Makes shows us how wood skewers are good for more than serving kebabs. By stacking, cutting, clamping, and gluing hundreds of sticks, then planing and sanding the result, he was able to create a beautiful cutting board we’d love to have on our counter.

via The Awesomer
Bamboo Skewer Chopping Board

How Ford Lost the First Mustang Ever Made—and What They Had to Do to Get it Back

Car fans, here’s a story you can tell your friends at the bar.

When Ford first designed their iconic Mustang, which was set to debut in April of 1964 as a model year 1965 car, they initially produced a limited batch to seed the dealerships with. This early batch—known as the 1964 1/2 model year among collectors—was intended both to familiarize the production line workers with the car, and to ensure that every major dealership in the U.S. and Canada had a display model in advance of the car’s launch.

In order to have all of the deliveries arrive in advance of the launch, the first car off of the line in February of 1964—serial #5F08F 00001—was shipped to the physically furthest dealership, a remote Ford outpost in St. John’s, Newfoundland.

These preproduction cars were not supposed to be sold. But a young pilot, Captain Stanley Tucker (below), spotted the display Mustang at the Newfoundland dealership and somehow convinced them—three days before launch, no less–to sell the car to him. Money talks, and the car changed hands.

Ford sold a whopping 680,000 Mustangs in their first year. Realizing they had a monster hit on their hands, the company sought to get serial #5F08F 00001 back, to preserve it for posterity. After learning that #5F08F 00001 had been sold, Ford contacted Captain Tucker directly and offered to buy it back.

Captain Tucker refused to sell it. 

Finally, after two years of strong (and presumably profitable) Mustang sales, in order to get #5F08F 00001 back, Ford made Captain Tucker an offer he couldn’t refuse:


via Core77
How Ford Lost the First Mustang Ever Made—and What They Had to Do to Get it Back

How to make your own bootable macOS 10.14 Mojave USB install drive


Article intro image
Enlarge /

It’s still pretty easy to make a bootable USB install drive for macOS Mojave.

Andrew Cunningham


via Ars Technica
How to make your own bootable macOS 10.14 Mojave USB install drive

SQL Order of Operations – SQL Query Order of Execution


Knowing the bits and bytes of an SQL query’s order of operations can be very valuable, as it can ease the process of writing new queries, while also being very beneficial when trying to optimize an SQL query.

If you’re looking for the short version, this is the logical order of operations, also known as the order of execution, for an SQL query:

You can speed up any SQL query, automatically. Try it now for free.

  1. FROM, including JOINs
  2. WHERE
  3. GROUP BY
  4. HAVING
  5. WINDOW functions
  6. SELECT
  7. DISTINCT
  8. UNION
  9. ORDER BY
  10. LIMIT and OFFSET

But the reality isn’t that easy nor straight forward. As we said, the SQL standard defines the order of execution for the different SQL query clauses. Said that, modern databases are already challanaging that default order by applying some optimization tricks which might change the actual order of execution, though they must end up returning the same result as if they were running the query at the default execution order.

Why would they do that? Well, it can be silly if the database would first fetch all data mentioned in the FROM clause (including the JOINs), before looking into the WHERE clause and its indexes. Those tables can hold lots of data, so you can imagine what will happen if the database’s optimizer would stick to the traditional order of operations of an SQL query.

Let’s look into each of the SQL query parts according to their execution order.

FROM and JOINs

The tables specified in the FROM clause (including JOINs), will be evaluated first, to determine the entire working set which is relevant for the query. The database will merge the data from all tables, according to the JOINs ON clauses, while also fetching data from the subqueries, and might even create some temporary tables to hold the data returned from the subqueries in this clause.

In many cases though, the database’s optimizer will choose to evaluate the WHERE part first, to see which part of the working set can be left out (preferably using indexes), so it won’t inflate the data set too much if it doesn’t really have to.

WHERE clause

The WHERE clause will be the second to be evaluated, after the FROM clause. We have the working data set in place, and now we can filter the data according to the conditions in the WHERE clause.

These conditions can include references to the data and tables from the FROM clause, but cannot include any references to aliases defined in the SELECT clause, as that data and those aliases may not yet ‘exist’ in that context, as that clause wasn’t yet evaluated by the database.

Also, a common pitfall for the WHERE clause would be to try and filter out aggregated values in the WHERE clause, for example with this clause: “WHERE sum(available_stock) > 0“. This statement will fail the query execution, because aggregations will be evaluated later in the process (see the GROUP BY section below). To apply filtering condition on aggregated data, you should use the HAVING clause and not the WHERE clause.

GROUP BY clause

Now that we filtered the data set using the WHERE clause, we can aggregate the data according to one or more columns appearing in the GROUP BY clause. Grouping the data is actually splitting it to different chunks or buckets, where each bucket has one key and a list of rows that match that key. Not having a GROUP BY clause is like putting all rows in a one huge bucket.

Once you aggregate the data, you can now use aggregation functions to return a per-group value for each of the buckets. Such aggregation functions include COUNT, MIN, MAX, SUM and others.

You can speed up any SQL query, automatically. Try it now for free.

HAVING clause

Now that we have grouped the data using the GROUP BY clause, we can use the HAVING clause to filter out some buckets. The conditions in the HAVING clause can refer to the aggregation functions, so the example which didn’t work in the WHERE clause above, will work just fine in the HAVING clause: “HAVING sum(available_stock) > 0″.

As we’ve already grouped the data, we can no longer access the original rows at this point, so we can only apply conditions to filter entire buckets, and not single rows in a bucket.

Also, as we mentioned in previous sections, aliases defined in the SELECT clause cannot be accessed in the section either, as they weren’t yet evaluated by the database (this is true in most databases).

Window functions

If you are using Window functions, this is the point where they’ll be executed. Just like the grouping mechanism, Window functions are also performing a calculation on a set of rows. The main difference is that when using Window functions, each row will keep its own identity and won’t be grouped into a bucket of other similar rows.

Window functions can only be used in either the SELECT or the ORDER BY clause. You can use aggregation functions inside the Window functions, for example:

SUM(COUNT(*)) OVER ()

SELECT clause

Now that we are done with discarding rows from the data set and grouping the data, we can select the data we want to be fetched from the query to the client side. You can use column names, aggregations and subqueries inside the SELECT clause. Keep in mind that if you’re using a reference to an aggregation function, such as COUNT(*) in the SELECT clause, it’s merely a reference to an aggregation which already occurred when the grouping took place, so the aggregation itself doesn’t happen in the SELECT clause, but this is only a reference to its result set.

DISTINCT keyword

The syntax of the DISTINCT keyword is a bit confusing, because the keyword takes its place before the column names in the SELECT clause. But, the actual DISTINCT operation takes place after the SELECT. When using the DISTINCT keyword, the database will discard rows with duplicate values from the remaining rows left after the filtering and aggregations took place.

UNION keyword

The UNION keyword combines the result sets of two queries into one result set. Most databases will allow you to choose between UNION DISTINCT (which will discard duplicate rows from the combined result set) or UNION ALL (which just combines the result sets without applying any duplication check).

You can apply sorting (ORDER BY) and limiting (LIMIT) on the UNION’s result set, the same way you can apply it on a regular query.

ORDER BY clause

Sorting takes place once the database has the entire result set ready (after filtering, grouping, duplication removal). Once we have that, the database can now sort the result set using columns, selected aliases, or aggregation functions, even if they aren’t part of the selected data. The only exception is when using the DISTINCT keyword, which prevents sorting by a non-selected column, as in that case the result set’s order will be undefined.

You can choose to sort the data using a descending (DESC) order or an ascending (ASC) order. The order can be unique for each of the order parts, so the following is valid: ORDER BY firstname ASC, age DESC

LIMIT and OFFSET

In most use cases (excluding a few like reporting), we would want to discard all rows but the first X rows of the query’s result. The LIMIT clause, which is executed after sorting, allows us to do just that. In addition, you can choose which row to start fetching the data from and how many to exclude, using a combination of the LIMIT and OFFSET keywords. The following example will fetch 50 rows starting row #100: LIMIT 50 OFFSET 100


via Planet MySQL
SQL Order of Operations – SQL Query Order of Execution