Oh, few issues push buttons quite like the Second Amendment, which, in truth, shouldn’t push buttons at all, right? It’s in black and white as Constitutional law, and the precedence for such a right is undeniable. But still, people bicker about it, split hairs, and grasp at straws to argue for limitations and modifications to this fundamental right. Why would this time be any different?
Enter from stage right this dude! Who believes limitations on God-given rights, like those enumerated in the Constitution, are sometimes necessary for the good of society—sound a little familiar?
“I don’t believe that I should be able to go buy a tank. That’s promotion of general welfare. That’s domestic tranquility. […] So, in my mind, the government should and ought to limit certain things. For example, I should not be able to buy a tank.”
To each their own, I suppose. I’d love to be able to buy a tank, and I do think people should be able to buy tanks. Maybe I’m in the minority.
You wrote a Python script that you’re proud of, and now you want to show it off to the world. But how? Most people won’t know what to do with your .py file. Converting your script into a Python web application is a great solution to make your code usable for a broad audience.
In this course, you’ll learn how to go from a local Python script to a fully deployed Flask web application that you can share with the world.
By the end of this course, you’ll know:
What web applications are and how you can host them online
How to convert a Python script into a Flask web application
How to improve user experience by adding HTML to your Python code
How to deploy your Python web application to Google App Engine
[ Improve Your Python With ð Python Tricks ð â Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
You wrote a Python script that you’re proud of, and now you want to show it off to the world. But how? Most people won’t know what to do with your .py file. Converting your script into a Python web application is a great solution to make your code usable for a broad audience.
In this course, you’ll learn how to go from a local Python script to a fully deployed Flask web application that you can share with the world.
By the end of this course, you’ll know:
What web applications are and how you can host them online
How to convert a Python script into a Flask web application
How to improve user experience by adding HTML to your Python code
How to deploy your Python web application to Google App Engine
[ Improve Your Python With ð Python Tricks ð â Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
The Pandas DataFrame has several methods concerning Computations and Descriptive Stats. When applied to a DataFrame, these methods evaluate the elements and return the results.
Part 1 focuses on the DataFrame methods abs(), all(), any(), clip(), corr(), and corrwith().
Part 2 focuses on the DataFrame methods count(), cov(), cummax(), cummin(), cumprod(), cumsum().
Part 3 focuses on the DataFrame methods describe(), diff(), eval(), kurtosis().
Part 4 focuses on the DataFrame methods mad(), min(), max(), mean(), median(), and mode().
Part 5 focuses on the DataFrame methods pct_change(), quantile(), rank(), round(), prod(), and product().
Getting Started
Remember to add the Required Starter Code to the top of each code snippet. This snippet will allow the code in this article to run error-free.
Required Starter Code
import pandas as pd
import numpy as np
Before any data manipulation can occur, two new libraries will require installation.
The pandas library enables access to/from a DataFrame.
The numpy library supports multi-dimensional arrays and matrices in addition to a collection of mathematical functions.
To install these libraries, navigate to an IDE terminal. At the command prompt ($), execute the code below. For the terminal used in this example, the command prompt is a dollar sign ($). Your terminal prompt may be different.
$ pip install pandas
Hit the <Enter> key on the keyboard to start the installation process.
$ pip install numpy
Hit the <Enter> key on the keyboard to start the installation process.
Feel free to check out the correct ways of installing those libraries here:
If the installations were successful, a message displays in the terminal indicating the same.
DataFrame pct_change()
The pct_change() method calculates and returns the percentage change between the current and prior element(s) in a DataFrame. The return value is the caller.
To fully understand this method and other methods in this tutorial from a mathematical point of view, feel free to watch this short tutorial:
If zero (0) or index, apply the function to each column. Default is None. If one (1) or column, apply the function to each row.
method
Determines how to rank identical values, such as: – The average rank of the group. – The lowest (min) rank value of the group. – The highest (max) rank value of the group. – Each assigns in the same order they appear in the array. – Density increases by one (1) between the groups.
numeric_only
Only include columns that contain integers, floats, or boolean values.
na_option
Determines how NaN values rank, such as: – Keep assigns a NaN to the rank values. – Top: The lowest rank to any NaN values found. – Bottom: The highest to any NaN values found.
ascending
Determines if the elements/values rank in ascending or descending order.
pct
If set to True, the results will return in percentile form. By default, this value is False.
For this example, a CSV file is read in and is ranked on Population and sorted. Click here to download and move this file to the current working directory.
The Pandas DataFrame has several methods concerning Computations and Descriptive Stats. When applied to a DataFrame, these methods evaluate the elements and return the results.
Part 1 focuses on the DataFrame methods abs(), all(), any(), clip(), corr(), and corrwith().
Part 2 focuses on the DataFrame methods count(), cov(), cummax(), cummin(), cumprod(), cumsum().
Part 3 focuses on the DataFrame methods describe(), diff(), eval(), kurtosis().
Part 4 focuses on the DataFrame methods mad(), min(), max(), mean(), median(), and mode().
Getting Started
Remember to add the Required Starter Code to the top of each code snippet. This snippet will allow the code in this article to run error-free.
Required Starter Code
import pandas as pd
import numpy as np
Before any data manipulation can occur, two new libraries will require installation.
The pandas library enables access to/from a DataFrame.
The numpy library supports multi-dimensional arrays and matrices in addition to a collection of mathematical functions.
To install these libraries, navigate to an IDE terminal. At the command prompt ($), execute the code below. For the terminal used in this example, the command prompt is a dollar sign ($). Your terminal prompt may be different.
$ pip install pandas
Hit the <Enter> key on the keyboard to start the installation process.
$ pip install numpy
Hit the <Enter> key on the keyboard to start the installation process.
Feel free to check out the correct ways of installing those libraries here:
Line [1] creates a DataFrame from a Dictionary of Lists and saves it to df_teams.
Line [2] uses the mad() method with the axis parameter set to columns to calculate MAD from the DataFrame. The lambda function formats the output to three (3) decimal places. This output saves to the result variable.
Line [1] creates a DataFrame from a dictionary of lists and saves it to df_teams.
Line [2] uses the min() method with the axis parameter set to columns to retrieve the minimum value(s) from the DataFrame. This output saves to the result variable.
Line [3] outputs the result to the terminal.
Output:
Bruins
4
Oilers
3
Leafs
2
Flames
8
dtype:
int64
This example uses two (2) arrays and retrieves the minimum value(s) of the Series.
Line [1] creates a DataFrame from a Dictionary of Lists and saves it to df_teams.
Line [2] uses max() with the axis parameter set to columns to retrieve the maximum value(s) from the DataFrame. This output saves to the result variable.
Line [3] outputs the result to the terminal.
Output:
Bruins
9
Oilers
14
Leafs
11
Flames
21
dtype:
int64
This example uses two (2) arrays and retrieves the maximum value(s) of the Series.
Line [1-2] create lists of random grades and assigns them to the appropriate variable.
Line [3] uses the NumPy library maximum function to compare the two (2) arrays. This output saves to the result variable.
Line [4] outputs the result to the terminal.
Output:
[73 84 83 93]
DataFrame mean()
The mean() method returns the average of the DataFrame/Series across a requested axis. If a DataFrame is used, the results will return a Series. If a Series is used, the result will return a single number (float).
Line [1] creates a DataFrame from a Dictionary of Lists and saves it to df_teams.
Line [2] uses the mean() method with the axis parameter set to columns to calculate means (averages) from the DataFrame. The lambda function formats the output to two (2) decimal places. This output saves to the result variable.
Line [3] outputs the result to the terminal.
Output:
Bruins
6.00
Oilers
7.67
Leafs
6.67
Flames
12.00
dtype:
float64
For this example, Alice Accord, an employee of Rivers Clothing has logged her hours for the week. Let’s calculate the mean (average) hours worked per day.
Code Example 2:
hours = pd.Series([40.5, 37.5, 40, 55])
result = hours.mean()
print(result)
Line [1] creates a Series of hours worked for the week and saves to hours.
Line [2] uses the mean() method to calculate the mean (average). This output saves to the result variable.
Line [3] outputs the result to the terminal.
Output:
42.25
DataFrame median()
The median() method calculates and returns the median of DataFrame/Series elements across a requested axis. In other words, the median determines the middle number(s) of the dataset.
To fully understand median from a mathematical point of view, watch this short tutorial:
If you’ve been spending more time at home during the pandemic, you may not realize how important proper footwear is for keeping certain injuries at bay.
Sean Peden, an orthopaedic foot and ankle specialist from Yale University Medicine, says not wearing supportive footwear on a regular basis can lead to foot pain and other problems.
“Many people are continuing to work at home part- or full-time, which for some can mean wearing slippers or walking around barefoot,” Peden says. “And because of that, many patients are coming to us with foot problems.”
Taking good care of your feet will not only help you avoid common injuries like tendonitis and plantar fasciitis, but it can also prevent other issues with your hips, knees, and back from developing, he adds.
Here, Peden shares some of the most common foot problems he sees—and simple treatments to get relief:
Think of shoes as shock absorbers
Just as you would pick out an appropriate shoe for your commute into the office, it’s important to put the same level of thought into selecting an at-home shoe.
Walking barefoot at home is not recommended for the same reason walking barefoot outside is ill-advised, Peden says.
“All kinds of footwear protect your feet. Over the course of weeks or months, the strain of walking barefoot can add significant stress to your arches, tendons, plantar fascia, and joints,” he says. “This can lead to a range of complications, from minor conditions such as calluses to major issues such as arch collapse.”
It may help to think of footwear as shock absorbers and, based on body type and gait, some of us need more shock absorption than others, Peden says.
“If you have sore feet—or have had foot problems in the past—wearing a pair of what I call ‘house shoes,’ or ‘house slippers,’ is a good idea.”
By that, Peden means a hard-soled, slip-on shoe or slipper that is worn exclusively inside the home (ideally) to avoid bringing in dirt or bacteria.
“To be practical, I suggest a slip-on clog or slipper without laces. That way, you don’t have to tie and untie your shoes 10 times a day,” Peden says. “A hard sole is important because the harder the sole, the less stress the joints and tendons in your foot experience with each step. The hard sole transfers that stress to the shoe rather than to the foot.”
In general, avoid fluffy, formless slippers, he advises. “If you are at home, you might go up and down stairs dozens of times a day—or do chores around the house. And those are not activities to do with footwear that doesn’t have any support,” Peden says. “A good rule of thumb is if it isn’t something you could walk in for a few blocks comfortably, you shouldn’t wear it around the house all day, either.”
Painful tendonitis
One of the most common foot problems Peden has seen in patients since the pandemic started is Achilles tendonitis, or inflammation of a tendon (a thick cord of tissue that connects muscles to bones). The Achilles tendon runs from the back of your calf to your heel bone. Achilles tendonitis can cause pain and swelling in the foot and ankle.
An injury, overuse, and flat fleet are all causes for Achilles tendonitis, Peden says. “It can be an issue especially if people with flat feet spend six months to a year not wearing supportive shoes on a regular basis,” he says. “The tendon in the arch of the foot becomes inflamed as the foot gets flatter. It is quite painful and can be debilitating.”
Peden says he is also seeing more patients with posterior tibial tendonitis, which causes a collapsed arch or flat foot.
The Fix: For acute pain, the first things to try are rest, ice, and staying off your feet as much as possible. Finding footwear with good arch support is another must, Peden says.
“Some people might need an ankle brace or additional inserts for their shoes, but for the vast majority, proper footwear is the answer. These tendon flares generally last a few months, but patients usually see improvement within a week or two.”
People with tendon issues should get proper treatment, Peden says. “You want to avoid developing a chronic tendon issue, because those are harder to cure.”
Plantar fasciitis: ‘stabbing’ heal pain
Many patients have developed plantar fasciitis, inflammation of the band of tissue on the bottom of your foot.
A common symptom is a stabbing pain in the heel that can be the most intense when you first step out of bed in the morning. That’s because the plantar fascia, which runs from the heel to the base of your toes, tightens overnight.
The plantar fascia supports the arch of the foot and absorbs stress. Too much stress—from standing on your feet on a hard surface for a long time, improper shoes, or running—can cause irritation and tiny tears in the band of tissue.
“The pain is usually on the bottom part of the heel,” Peden says. “It’s associated with tight Achilles tendons and calf muscles. If people spend a lot of their day sitting, for example, the muscles can tighten up, and wearing improper footwear can exacerbate the issue.
“For people who work outside the home and are on their feet all day, including nurses, they should wear a supportive shoe—and not something too soft or flexible. This can include sneakers, a hard clog, or a work shoe, depending on personal preference.”
The Fix: Besides supportive footwear and avoiding walking around barefoot, treatment should include a home stretching program to address the tightness in the calf muscles and Achilles tendons, Peden says.
Another effective treatment is to wear a soft, flexible splint that holds your foot at a 90-degree angle while you are sleeping; this keeps the plantar fascia stretched out. You can also wear a splint while lying on the couch watching TV.
As painful as plantar fasciitis can be, Peden says it is not a progressive condition. “People often worry it’s the start of something like arthritis, which continues to get worse,” he says. “It might take a few months of conservative, noninvasive, nonsurgical treatments, but patients with plantar fasciitis typically get better.”
Physical therapy and lifestyle
Exercise, physical therapy, and weight loss can all make a difference in addressing foot pain, too.
“One pound of additional weight on your body leads to six pounds of additional pressure on your foot. So, if you lose 10 pounds, that is really taking 60 pounds of pressure off your foot,” Peden says.
With the pandemic, many people have gained weight, which compounds the problem. But the key is not to do too much too quickly to try to reverse it, Peden says.
“If you try to lose weight by suddenly walking too much, that’s hard on your feet, too, and may lead to other foot problems. So, I often recommend cross-training, including low-impact cardio activities like biking or swimming. You can walk, but try to take it easy and, as always, wear good, supportive shoes.”
Hiking shoes are often a good option, particularly if you walk on uneven surfaces, including trails. “They are a little safer than sneakers, and protect your foot and ankle better,” he says.
In certain cases, physical therapy is recommended for lingering foot issues. “Physical therapists have many techniques that can speed up the recovery process,” Peden says.
Surgery is rarely needed for chronic conditions like tendonitis or plantar fasciitis. “We always treat our patients first with nonsurgical options to hopefully manage the condition before we ever talk about surgery,” Peden says.
But if you are feeling foot pain, don’t be afraid to seek medical help, Peden advises.
“I know people have different comfort levels right now about seeking medical care during the pandemic, but if you have a foot issue and it’s been hurting for a while, you should go see your doctor. There are likely easy solutions.”
In this blog post, we will compare the performance of performing a backup from a MySQL database using mysqldump, MySQL Shell feature called Instance Dump, mysqlpump, mydumper, and Percona XtraBackup. All these available options are open source and free to use for the entire community.
To start, let’s see the results of the test.
Benchmark Results
The benchmark was run on an m5dn.8xlarge instance, with 128GB RAM, 32 vCPU, and 2xNVMe disks of 600GB (one for backup and the other one for MySQL data). The MySQL version was 8.0.26 and configured with 89Gb of buffer pool, 20Gb of redo log, and a sample database of 177 GB (more details below).
We can observe the results in the chart below:
And if we analyze the chart only for the multi-threaded options:
As we can see, for each software, I’ve run each command three times in order to experiment using 16, 32, and 64 threads. The exception for this is mysqldump, which does not have a parallel option and only runs in a single-threaded mode.
We can observe interesting outcomes:
When using zstd compression, mydumper really shines in terms of performance. This option was added not long ago (MyDumper 0.11.3).
When mydumper is using gzip, MySQL Shell is the fastest backup option.
In 3rd we have Percona XtraBackup.
mysqlpump is the 4th fastest followed closer by mydumper when using gzip.
mysqldump is the classic old-school style to perform dumps and is the slowest of the four tools.
In a server with more CPUs, the potential parallelism increases, giving even more advantage to the tools that can benefit from multiple threads.
Before starting the comparison, I ran mysqldump once and discarded the results to warm up the cache, otherwise our test would be biased because the first backup would have to fetch data from the disk and not the cache.
With everything set, I started the mysqldump with the following options:
PS: To use zstd, there are no changes in the command line, but you need to download the zstd binaries.
For mysqlpump:
$ time mysqlpump --default-parallelism=16 --all-databases > backup.out
For xtrabackup:
$ time xtrabackup --backup --parallel=16 --compress --compress-threads=16 --datadir=/mysql_data/ --target-dir=/backup/
Analyzing the Results
And what do the results tell us?
Parallel methods have similar performance throughput. The mydumper tool cut the execution time by 50% when using zstd instead of gzip, so the compression method makes a big difference when using mydumper.
For the util.dumpInstance utility, one advantage is that the tool stores data in both binary and text format and uses zstd compression by default. Like mydumper, it uses multiple files to store the data and has a good compression ratio.
XtraBackup got third place with a few seconds of difference from MySQL shell. The main advantage of XtraBackup is its flexibility, providing PITR and encryption for example.
Next, mysqlpump is more efficient than mydumper with gzip, but only by a small margin. Both are logical backup methods and works in the same way. I tested mysqlpump with zstd compression, but the results were the same, hence the reason I didn’t add it to the chart. One possibility is because mysqlpump streams the data to a single file.
Lastly, for mysqldump, we can say that it has the most predictable behavior and has similar execution times with different runs. The lack of parallelism and compression is a disadvantage for mysqldump; however, since it was present in the earliest MySQL versions, based on Percona cases, it is still used as a logical backup method.
Please leave in the comments below what you thought about this blog post, if I missed something, or if it helped you. I will be glad to discuss it!
Useful Resources
Finally, you can reach us through the social networks, our forum, or access our material using the links presented below:
Andre Antunes improves another viral video by adding a heavy metal soundtrack to it. This time, he amped up the already formidable footage of New Zealand’s All Blacks rugby team performing a traditional Maori war dance known as the Haka. Original video here.
The path to becoming a proficient and successful programmer is difficult, but one that is certainly achievable. Data structures are a core component that every programming student must master, and chances are you may have already learned or worked with some basic data structures such as arrays or lists.
Interviewers tend to prefer asking questions related to data structures, so if you’re preparing for a job interview, you’re going to need to brush up on your data structures knowledge. Read on as we list the most important data structures for programmers and job interviews.
1. Linked List
Linked lists are one of the most basic data structures and often the starting point for students in most data structures courses. Linked lists are linear data structures that allow sequential data access.
Elements within the linked list are stored in individual nodes that are connected (linked) using pointers. You can think of a linked list as a chain of nodes connected to each other via different pointers.
Before we get into the specifics of the different types of linked lists, it is crucial to understand the structure and implementation of the individual node. Each node in a linked list has at least one pointer (doubly linked list nodes have two pointers) that connects it to the next node in the list and the data item itself.
MAKEUSEOF VIDEO OF THE DAY
Each linked list has a head and tail node. Single-linked list nodes only have one pointer that points to the next node in the chain. In addition to the next pointer, doubly linked list nodes have another pointer that points to the previous node in the chain.
Interview questions related to linked lists usually revolve around the insertion, searching, or deletion of a specific element. Insertion in a linked list takes O(1) time, but deletion and searching can take O(n) time in the worst case. So linked lists are not ideal.
2. Binary Tree
Binary trees are the most popular subset of the tree family data structure; elements in a binary tree are arranged in a hierarchy. Other types of trees include AVL, red-black, B trees, etc. Nodes of the binary tree contain the data element and two pointers to each child node.
Each parent node in a binary tree can have a maximum of two child nodes, and each child node, in turn, can be a parent to two nodes.
A binary search tree (BST) stores data in a sorted order, where elements with a key-value less than the parent node are stored on the left, and elements with a key-value greater than the parent node are stored on the right.
Binary trees are commonly asked in interviews, so if you’re getting ready for an interview, you should know how to flatten a binary tree, look up a specific element, and more.
3. Hash Table
Hash tables or hash maps are a highly efficient data structure that stores data in an array format. Each data element is assigned a unique index value in a hash table, which allows for efficient searching and deletion.
The process of assigning or mapping keys in a hash map is called hashing. The more efficient the hash function, the better the efficiency of the hash table itself.
Each hash table stores data elements in a value-index pair.
Where value is the data to be stored, and index is the unique integer used for mapping the element into the table. Hash functions can be very complex or very simple, depending on the required efficiency of the hash table and how you will resolve collisions.
Collisions often arise when a hash function produces the same mapping for different elements; hash map collisions can be resolved in different ways, using open addressing or chaining.
Hash tables or hash maps have a variety of different applications, including cryptography. They are the first choice data structure when insertion or searching in constant O(1) time is required.
4. Stacks
Stacks are one of the simpler data structures and are pretty easy to master. A stack data structure is essentially any real-life stack (think of a stack of boxes or plates) and operates in a LIFO (Last In First Out) manner.
Stacks’ LIFO property means the element you inserted last will be accessed first. You cannot access elements below the top element in a stack without popping the elements above it.
Stacks have two primary operations—push and pop. Push is used to insert an element into the stack, and pop removes the topmost element from the stack.
They also have plenty of useful applications, so it is very common for interviewers to ask questions related to stacks. Knowing how to reverse a stack and evaluate expressions is quite essential.
Queues are similar to stacks but operate in a FIFO (First In First Out) manner, meaning you can access the elements you inserted earlier. The queue data structure can be visualized as any real-life queue, where people are positioned based on their order of arrival.
Insertion operation of a queue is called enqueue, and deleting/removing an element from the beginning of the queue is referred to as dequeuing.
Priority queues are an integral application of queues in many vital applications such as CPU scheduling. In a priority queue, elements are ordered according to their priority rather than the order of arrival.
6. Heaps
Heaps are a type of binary tree where nodes are arranged in ascending or descending order. In a Min Heap, the key value of the parent is equal to or less than that of its children, and the root node contains the minimum value of the entire heap.
Similarly, the root node of a Max Heap contains the maximum key value of the heap; you must retain the min/max heap property throughout the heap.
Heaps have plenty of applications due to their very efficient nature; primarily, priority queues are often implemented through heaps. They are also a core component in heapsort algorithms.
Learn Data Structures
Data structures can seem harrowing at first but devote enough time, and you’ll find them easy as pie.
They are a vital part of programming, and almost every project will require you to use them. Knowing what data structure is ideal for a given scenario is critical.
7 Algorithms Every Programmer Should Know
These algorithms are essential to every programmer’s workflow.
Read Next
About The Author
M. Fahad Khawaja (84 Articles Published)
Fahad is a writer at MakeUseOf and is currently majoring in Computer Science. As an avid tech-writer he makes sure he stays updated with the latest technology. He finds himself particularly interested in football and technology.
Note: The first line contains 
In this blog post, we will compare the performance of performing a backup from a MySQL database using 
