Month: July 2021

https://engineering.fb.com/wp-content/uploads/2021/07/CD21_390_ENG_MySQL_Animation_FINAL.gif?w=1024

MySQL, an open source database developed by Oracle, powers some of Facebook’s most important workloads. We actively develop new features in MySQL to support our evolving requirements. These features change many different areas of MySQL, including client connectors, storage engine, optimizer, and replication. Each new major version of MySQL requires significant time and effort to migrate our workloads. The challenges  include:

• Porting our custom features to the new version
• Ensuring replication is compatible between the major versions
• Minimizing changes needed for existing application queries
• Fixing performance regressions that prevent the server from supporting our workloads

Our last major version upgrade, to MySQL 5.6, took more than a year to roll out. When version 5.7 was released, we were still in the midst of developing our LSM-Tree storage engine, MyRocks, on version 5.6. Since upgrading to 5.7 while simultaneously building a new storage engine would have significantly slowed the progress on MyRocks, we opted to stay with 5.6 until MyRocks was complete. MySQL 8.0 was announced as we were finishing the rollout of MyRocks to our user database (UDB) service tier. 

That version included compelling features like writeset-based parallel replication and a transactional data dictionary that provided atomic DDL support. For us, moving to 8.0 would also bring in the 5.7 features we had missed, including Document Store. Version 5.6 was approaching end of life, and we wanted to stay active within the MySQL community, especially with our work on the MyRocks storage engine. Enhancements in 8.0, like instant DDL, could speed up MyRocks schema changes, but we needed to be on the 8.0 codebase to use it. Given the benefits of the code update, we decided to migrate to 8.0. We’re sharing how we tackled our 8.0 migration project — and some of the surprises we discovered in the process. When we initially scoped out the project, it was clear that moving to 8.0 would be even more difficult than migrating to 5.6 or MyRocks.

• At the time, our customized 5.6 branch had over 1,700 code patches to port to 8.0. As we were porting those changes, new Facebook MySQL features and fixes were added to the 5.6 codebase that moved the goalpost further away.
• We have many MySQL servers running in production, serving a large number of disparate applications. We also have extensive software infrastructure for managing MySQL instances. These applications perform operations like gathering statistics and managing server backups.
• Upgrading from 5.6 to 8.0 skipped over 5.7 entirely. APIs that were active in 5.6 would have been deprecated in 5.7 and possibly removed in 8.0, requiring us to update any application using the now-removed APIs.
• A number of Facebook features were not forward-compatible with similar ones in 8.0 and required a deprecation and migration path forward.
• MyRocks enhancements were needed to run in 8.0, including native partitioning and crash recovery.

Code patches

We first set up the 8.0 branch for building and testing in our development environments. We then began the long journey to port the patches from our 5.6 branch. There were more than 1,700 patches when we started, but we were able to organize them into a few major categories. Most of our custom code had good comments and descriptions so we could easily determine whether it was still needed by the applications or if it could be dropped. Features that were enabled by special keywords or unique variable names also made it easy to determine relevance because we could search through our application codebases to find their use cases. A few patches were very obscure and required detective work — digging through old design documents, posts, and/or code review comments — to understand their history.

We sorted each patch into one of four buckets:

1. Drop: Features that were no longer used, or had equivalent functionality in 8.0, did not need to be ported.
2. Build/Client: Non-server features that supported our build environment and modified MySQL tools like mysqlbinlog, or added functionality like the async client API, were ported.
3. Non-MyRocks Server: Features in the mysqld server that were not related to our MyRocks storage engine were ported.
4. MyRocks Server: Features that supported the MyRocks storage engine were ported.

We tracked the status and relevant historical information of each patch using spreadsheets, and recorded our reasoning when dropping a patch. Multiple patches that updated the same feature were grouped together for porting. Patches ported and committed to the 8.0 branch were annotated with the 5.6 commit information. Discrepancies on porting status would inevitably arise due to the large number of patches we needed to sift through and these notes helped us resolve them.

Each of the client and server categories naturally became a software release milestone. With all client-related changes ported, we were able to update our client tooling and connector code to 8.0. Once all of the non-MyRocks server features were ported, we were able to deploy 8.0 mysqld for InnoDB servers. Finishing up the MyRocks server features enabled us to update MyRocks installations.

Some of the most complex features required significant changes for 8.0, and a few areas had major compatibility problems. For example, upstream 8.0 binlog event formats were incompatible with some of our custom 5.6 modifications. Error codes used by Facebook 5.6 features conflicted with those assigned to new features by upstream 8.0. We ultimately needed to patch our 5.6 server to be forward-compatible with 8.0.

It took a couple of years to complete porting all of these features. By the time we got to the end, we had evaluated more than 2,300 patches and ported 1,500 of those to 8.0.

The migration path

We group together multiple mysqld instances into a single MySQL replica set. Each instance in a replica set contains the same data but is geographically distributed to a different data center to provide data availability and failover support. Each replica set has one primary instance. The remaining instances are all secondaries. The primary handles all write traffic and replicates the data asynchronously to all secondaries.

We started with replica sets consisting of 5.6 primary/5.6 secondaries and the end goal was replica sets with 8.0 primary/8.0 secondaries. We followed a plan similar to the UDB MyRocks migration plan.

1. For each replica set, create and add 8.0 secondaries via a logical copy using mysqldump. These secondaries do not serve any application read traffic.
2. Enable read traffic on the 8.0 secondaries.
3. Allow the 8.0 instance to be promoted to primary.
4. Disable the 5.6 instances for read traffic.
5. Remove all the 5.6 instances.

Each replica set could transition through each of the steps above independently and stay on a step as long as needed. We separated replica sets into much smaller groups, which we shepherded through each transition. If we found problems, we could rollback to the previous step. In some cases, replica sets were able to reach the last step before others started.

To automate the transition of a large number of replica sets, we needed to build new software infrastructure. We could group replica sets together and move them through each stage by simply changing a line in a configuration file. Any replica set that encountered problems could then be individually rolled back.

Row-based replication

As part of the 8.0 migration effort, we decided to standardize on using row-based replication (RBR). Some 8.0 features required RBR, and it simplified our MyRocks porting efforts. While most of our MySQL replica sets were already using RBR, those still running statement-based replication (SBR) could not be easily converted. These replica sets usually had tables without any high cardinality keys. Switching completely to RBR had been a goal, but the long tail of work needed to add primary keys was often prioritized lower than other projects.

Hence, we made RBR a requirement for 8.0. After evaluating and adding primary keys to every table, we switched over the last SBR replica set this year. Using RBR also gave us an alternative solution for resolving an application issue that we encountered when we moved some replica sets to 8.0 primaries, which will be discussed later.

Automation validation

Most of the 8.0 migration process involved testing and verifying the mysqld server with our automation infrastructure and application queries.

As our MySQL fleet grew, so did the automation infrastructure we use to manage the servers. In order to ensure all of our MySQL automation was compatible with the 8.0 version, we invested in building a test environment, which leveraged test replica sets with virtual machines to verify the behaviors. We wrote integration tests to canary each piece of automation to run on both the 5.6 version and the 8.0 version and verified their correctness. We found several bugs and behavior differences as we went through this exercise.

As each piece of MySQL infrastructure was validated against our 8.0 server, we found and fixed (or worked around) a number of interesting issues:

1. Software that parsed text output from error log, mysqldump output, or server show commands easily broke. Slight changes in the server output often revealed bugs in a tool’s parsing logic.
2. The 8.0’s default utf8mb4 collation settings resulted in collation mismatches between our 5.6 and 8.0 instances. 8.0 tables may use the new utf8mb4_0900 collations even for create statements generated by 5.6’s show create table because the 5.6 schemas using utf8mb4_general_ci do not explicitly specify collation. These table differences often caused problems with replication and schema verification tools.
3. The error codes for certain replication failures changed and we had to fix our automation to handle them correctly.
4. The 8.0 version’s data dictionary obsoleted table .frm files, but some of our automation used them to detect table schema modifications.
5. We had to update our automation to support the dynamic privs introduced in 8.0.

Application validation

We wanted the transition for applications to be as transparent as possible, but some application queries hit performance regressions or would fail on 8.0.

For the MyRocks migration, we built a MySQL shadow testing framework that captured production traffic and replayed them to test instances. For each application workload, we constructed test instances on 8.0 and replayed shadow traffic queries to them. We captured and logged the errors returning from the 8.0 server and found some interesting problems. Unfortunately, not all of these problems were found during testing. For example, the transaction deadlock was discovered by applications during the migration. We were able to roll back these applications to 5.6 temporarily while we researched different solutions.

• New reserved keywords were introduced in 8.0 and a few, such as groups and rank, conflicted with popular table column names and aliases used in application queries. These queries did not escape the names via backquotes, leading to parsing errors. Applications using software libraries that automatically escaped the column names in queries did not hit these issues, but not all applications used them. Fixing the problem was simple, but it took time to track down application owners and codebases generating these queries.
• A few REGEXP incompatibilities were also found between 5.6 and 8.0.
• A few applications hit repeatable-read transaction deadlocks involving insert … on duplicate key queries on InnoDB. 5.6 had a bug which was corrected in 8.0, but the fix increased the likelihood of transaction deadlocks. After analyzing our queries, we were able to resolve them by lowering the isolation level. This option was available to us since we had made the switch to row-based replication.
• Our custom 5.6 Document Store and JSON functions were not compatible with 8.0’s. Applications using Document Store needed to convert the document type to text for the migration. For the JSON functions, we added 5.6-compatible versions to the 8.0 server so that applications could migrate to the 8.0 API at a later time.

Our query and performance testing of the 8.0 server uncovered a few problems that needed to be addressed almost immediately.

• We found new mutex contention hotspots around the ACL cache. When a large number of connections were opened simultaneously, they could all block on checking ACLs.
• Similar contention was found with binlog index access when many binlog files are present and high binlog write rates rotate files frequently.
• Several queries involving temp tables were broken. The queries would return unexpected errors or take so long to run that they would time out.

Memory usage compared with 5.6 had increased, especially for our MyRocks instances, because InnoDB in 8.0 must be loaded. The default performance_schema settings enabled all instruments and consumed significant memory. We limited the memory usage by only enabling a small number of instruments and making code changes to disable tables that could not be manually turned off. However, not all the increased memory was being allocated by performance_schema. We needed to examine and modify various InnoDB internal data structures to reduce the memory footprint further. This effort brought 8.0’s memory usage down to acceptable levels. 

What’s next

The 8.0 migration has taken a few years so far. We have converted many of our InnoDB replica sets to running entirely on 8.0. Most of the remaining ones are at various stages along the migration path. Now that most of our custom features have been ported to 8.0, updating to Oracle’s minor releases has been comparatively easier and we plan to keep pace with the latest versions.

Skipping a major version like 5.7 introduced problems, which our migration needed to solve.

First, we could not upgrade servers in place and needed to use logical dump and restore to build a new server. However, for very large mysqld instances, this can take many days on a live production server and this fragile process will likely be interrupted before it can complete. For these large instances, we had to modify our backup and restore systems to handle the rebuild.

Second, it is much harder to detect API changes because 5.7 could have provided deprecation warnings to our application clients to fix potential issues. Instead, we needed to run additional shadow tests to find failures before we could migrate the production workloads. Using mysql client software that automatically escaped schema object names helps reduce the number of compatibility issues.

Supporting two major versions within a replica set is hard. Once a replica set promotes its primary to be an 8.0 instance, it is best to disable and remove the 5.6 ones as soon as possible. Application users tend to discover new features that are supported only by 8.0, like utf8mb4_0900 collations, and using these can break the replication stream between 8.0 and 5.6 instances.

Despite all the hurdles in our migration path, we have already seen the benefits of running 8.0. Some applications have opted for early conversion to 8.0 to utilize features like Document Store and improved datetime support. We have been considering how to support storage engine features like Instant DDL on MyRocks. Overall, the new version greatly expands on what we can do with MySQL @ Facebook.

The post Migrating Facebook to MySQL 8.0 appeared first on Facebook Engineering.

Planet MySQL

https://s.yimg.com/os/creatr-uploaded-images/2021-07/73d2f420-ea45-11eb-9e52-10f1e9322bff

After sharing a short teaser earlier in the month, Netflix has released a full trailer for Nightmare of the Wolf, its upcoming animated Witcher movie, and shared details on the voice cast. Theo James, known best for his leading role in the Divergent series, voices protagonist Vesemir, years before the Witcher became the mentor of Geralt of Rivia. 

James also voiced the character in season one of The Witcher. “Vesemir grew up as a servant on a nobleman’s estate, working long days for too little coin,” Netflix writes of the charming monster hunter. “He yearns to break free of his social standing and to explore the Continent to find his destiny.”

On his adventure, he’ll meet a sorceress named Tetra, who we see throughout the trailer. If Tetra’s voice sounds familiar, it’s because she’s voiced by Dota: Dragon’s Blood actor Lara Pulver. Rounding out the main cast are Mary McDonnell as Lady Zerbst, a Kaedweni noblewoman, and Graham McTavish as Deglan, a Witcher from the Skellige Isles. Notably, McTavish will play Redanian spymaster Dijkstra in season two of The Witcher.

The Witcher: Nightmare of the Wolf debuts on Netflix on August 28th. It will give fans of the fantasy franchise something to watch before season two of The Witcher starts streaming on December 17th.

Engadget

http://img.youtube.com/vi/n2WOag1G7Zg/0.jpgA demo project to show how Import works with Laravel Excel, and how to handle relationships rows.Laravel News Links

https://www.louderwithcrowder.com/media-library/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpbWFnZSI6Imh0dHBzOi8vYXNzZXRzLnJibC5tcy8yNzAyMTYwMi9vcmlnaW4uanBnIiwiZXhwaXJlc19hdCI6MTY0ODc5MjkyOX0.YyQoFWyYxTHJfvUayiPPeVRXH_FWR4IhHn7YJ1bTe9o/image.jpg?width=980

There’s one in every crowd. A deranged SJW interrupts Steven during a civil Change My Mind. Watch what happens next.

Steven VS. Deranged SJW! | Louder With Crowder

youtu.be

RELATED ARTICLES:

• Epic Self Own: Teacher Thinks Ron DeSantis Banning CRT is Literally ‘The Hunger Games’
• Anti-Ron DeSantis Meltdowns Have Begun on TikTok and This Woman Needs to Calm Down
• Teacher has Meltdown Over State Law Preventing Her from Indoctrinating Your Kids
• Woman Has Epic Elevator Meltdown Over Someone She Claims Isn’t Masked Up Properly

Need a quick laugh? Check out and subscribe to our CrowderBits YouTube channel for Louder with Crowder skits, opens, and parody videos!

Louder With Crowder

Let’s expand upon the code we wrote in the previous video and clean it up using custom…Laracasts

https://media.notthebee.com/articles/e9187714-4345-4876-b3a7-6c4dfc063115.jpg

YOWZA!

Not the Bee

https://www.buckeyefirearms.org/sites/buckeyefirearms.org/files/styles/slideshow/public/field/image/ODNR_2.jpg?itok=ZBBMRG1j

Buckeye Firearms Association

https://content.artofmanliness.com/uploads/2021/07/concrete-2.jpg

Concrete is the most widely used building material on earth. Humans have been using it for thousands of years and have built impressive structures with it that have stood the test of time. If you’re a homeowner, you’ll likely use concrete for some simple DIY jobs: setting a fence post or a basketball hoop, or laying a concrete slab for a patio. 

Despite the ubiquity of concrete, there’s a good chance you don’t know that much about it. Let’s remedy that today by going over a few basics about this building material.  

What Is Concrete Made Of?

Concrete is a composite material made of three main ingredients: cement, aggregate, and water. 

Cement. People often use the words “concrete” and “cement” interchangeably, but they’re not the same thing. Cement is a component of concrete. It binds together the aggregate (more on that in a bit) and, when combined with water, gives concrete its solid and durable properties. 

Most modern cement is Portland cement. To make Portland cement, cement manufacturers combine limestone with a silica source (like slag, fly ash, or clay) in a giant kiln. They then heat up the mixture to temperatures as high as 3,000 degrees Fahrenheit. This drives all the water out of the mix as well as the carbon dioxide from the limestone. The heat chemically mixes all the ingredients together to form a new compound called clinker. The clinker is then ground to a very fine powder and mixed with gypsum. 

Boom. Portland cement. 

Aggregate. Cement is rarely used on its own. You typically mix it with some sort of aggregate. Aggregate can be sand and small rocks. Cement mixed with sand gives you mortar to lay a brick wall. Cement mixed with sand and epoxies gives you grout to fill the gap between tiles. Cement mixed with sand and small ground-up stones (up to an inch in diameter) gives you concrete. 

The ready-mix bags of concrete that you buy at Home Depot already have cement and aggregate combined. You just need to add . . .

Water. Water is the third and final ingredient of concrete. When you add water to your cement and aggregate mix, a chemical reaction occurs between the components called “hydration” or “curing.” 

The hydration process results in calcium silica hydrate (CSH). CSH hardens and interlocks to give concrete its solid and durable nature. In short, water is what makes concrete hard. 

Understanding that water is what kickstarts a chemical reaction in cement to give concrete its hardness should clear up a popular (yet erroneous) belief that concrete hardens by drying out. In fact, if your newly laid concrete slab dries out too fast, you’ll end up with a weakened concrete slab. This is why, as we’ll see below, properly laying concrete requires that you keep it moist for the first few days, as this is when most of the curing takes place. 

Concrete Mixing Tips

Mixing concrete isn’t difficult. For most DIY projects around the home, you can simply use bags of the just-add-water concrete mix you can buy at Home Depot or Lowes.

It’s easiest to mix concrete in a wheelbarrow. The wheelbarrow’s ample tub makes mixing large amounts of concrete and moving your concrete mix to where you need it much easier. But you can also use a bucket or some other tub if that’s all you have. 

First dump the dry concrete mix into your wheelbarrow or tub and push it all the way to one side. 

Now comes the tricky part: Knowing how much water to put in: Too little and you won’t get the hydration process going; too much, and you’ll get a soupy mix that will result in weakened concrete. 

Your ready-mix bag should tell you how much water you need to add, but you also need to apply your own eyeballs and judgement to this general recommendation. Fill a bucket with the recommended amount of water and slowly pour some of it into your dry concrete mix in the wheelbarrow. Don’t add all of the water right away. After you’ve poured some water in, mix it with the concrete mix using a hoe. Pour some more water in. Mix with the hoe. Pour some more water. Mix. Repeat the process until you’ve got a mix that’s neither too dry nor too wet. 

Concrete that’s too dry will look like clumpy balls. Concrete that’s too wet will look really soupy. Just-right concrete is pliable but not runny. It looks like a thick porridge. 

Typically, concrete starts curing a few hours after mixing, so you should use your newly-created concrete within that time frame. For larger projects, you’ll want to look into renting a concrete mixer. This will not only make mixing large amounts of concrete easier, but the constant rotation of the mixer will keep the concrete in its liquid form and prevent curing from starting. 

Be sure to clean up your wheelbarrow and tools as soon as you’re done with your job. You don’t want the concrete to cure and stick to them. The easiest way to get the job done is to rinse the concrete off with water and scrub with a stiff brush. As long as you’ve diluted the remaining concrete residue with plenty of water, it’s okay to pour it out in an inconspicuous place in your yard. Some people also just dig a hole, pour the residue in there, and then cover with dirt. Concrete is mostly limestone, so you’re just adding limestone to the ground. Won’t hurt anything as long as it’s not large quantities and it’s thoroughly diluted. 

Keeping Your Newly-Poured Concrete Damp 

Concrete does most of its curing in the first few days. As mentioned above, water is a necessary component of that curing process. That’s why it’s a good idea to keep your newly-poured concrete damp for a few days after you’ve poured it. Simply mist it with a hose. That’s it. 

Weather and Concrete

Because concrete requires water to cure and harden, you’ll need to consider the weather when working with concrete. If it’s too hot outside, the water in the freshly-made concrete can evaporate too quickly for the hydration process to occur. If it’s too cold, the water can freeze, disrupting the curing process. 

Ideally, work with your concrete when the temperature is between 40 degrees and 90 degrees Fahrenheit. You’re less likely to have issues within that range. 

With that said, you can still work with concrete above and below that range; you’ll just need to take extra precautions. If it’s really hot outside, be especially vigilant about keeping your concrete damp. 

If it’s freezing outside, you can add a chemical accelerant to your concrete so that the curing process speeds up. You can also keep heaters near your concrete or put tarps over your newly poured concrete to keep it warm.

The above is a basic intro on how concrete works and how to mix it. You’ll be able to set a post or basketball hoop or even make some concrete weights with the above instructions. 

If you’re looking to pour a concrete slab or make concrete stairs, you’ll want to check out all the fantastic video tutorials that exist on YouTube. 

At the least, you now know not to call concrete cement. 

The post A Beginner’s Guide to Concrete appeared first on The Art of Manliness.

The Art of Manliness

https://assets.amuniversal.com/66b1b810c0a20139694a005056a9545d

Dilbert Daily Strip

https://theawesomer.com/photos/2021/07/the_print_shop_online_t.jpg

Link

’80s kids might remember a little computer program called The Print Shop. Broderbund’s whiz-bang piece of software let users print out greeting cards, banners, and signs on dot-matrix printers. Now you can relive this classic thanks to Melody and April Ayres-Griffiths online emulation, complete with the ability to print to PDFs.

The Awesomer