Skip to main content

Posts

Showing posts from 2022

Standard Normal Distribution with examples using Python

Easy Guide to using f-strings in Python with examples

Easy Guide to using f-strings in Python with examples f-strings were introduced in Python 3.6. String formatting has been made simpler with f-strings. f-strings are of the form f" " or f' ' . Let's look at some examples to understand this better. How to print a variable using f-strings? name = "James Bond" drink = "martini" serving_type = "shaken and not stirred" f"My name is {name} and I like my {drink} {serving_type}" --->My name is James Bond and I like my martini shaken and not stirred f-strings use a { } to evaluate any expression and format it as a string. The above example evaluates each of the variables within curly braces and formats it as string. How to use mathematical expressions with f-strings f"2x2 = {2*2}" --->2x2 = 4 f"Square of 8 is {8**2} ---> Square of 8 is 64 Practically anything can be put inside the { } as long as the expresson eval

How to format float and numerical values using f-strings

f-strings have made life easier for Python developers. They are of the form f" {} " where the curly brackets are optional. The advantage of curly brackets is that expressions can be evaluated within f-strings at runtime and printed to the screen. We need not cast the variable to string format anymore. They have been around since Python 3.6 How to restrict decimal to n positions for a float variable? pi= 22/7 f"Value of pi upto 2 decimal positions is {pi:0.2f}" --->Value of pi upto 2 decimal positions is 3.14 f"Value of pi upto 3 decimal positions is {pi:0.3f}" --->Value of pi upto 3 decimal positions is 3.143 f"Value of pi upto 4 decimal positions is {pi:0.4f}" --->Value of pi upto 4 decimal positions is 3.1429 f"Value of pi with no decimal position is {pi:0.0f}" --->Value of pi with no decimal position is 3 How to format a fraction as a Percentage using f-strings? perc = 0.555553

How to adopt Embeddings for Categorical features in Tabular Data using PyTorch's nn.Embedding( )-- Part 2

In the previous post , we set up the context to utilize embeddings for categorical features. In this post, we will figure out how to create these embeddings and combine them with other continuous features to build a neural network model. Dataset Download We will utilize the UCI machine learning repo which has a dataset on credit card default for customers in Taiwan. This dataset is also available in Kaggle . Metadata about this dataset is available on the respective websites. To follow this post, it is recommended to download the dataset from Kaggle. Most of the features are self explanatory. Embedding Creation A few definitions first. Levels in a categorical feature represent unique values available for that categorical feature. For e.g. MARRIAGE has levels 0,1,2,3. Each level of a categorical feature is represented by a vector of numbers. So, if you stack up all the levels together and all the vectors together, you can imagine levels to be a colum

How to adopt Embeddings for Categorical features in Tabular Data using PyTorch's nn.Embedding( )-- Part 1

How to adopt Embeddings for Categorical features in Tabular Data using PyTorch's nn.Embedding( )-- Part 1 In this post, we will talk about using embeddings for categorical features using PyTorch. This post will be broken down into following parts. Dataset Download Data Understanding Data Preprocessing Embedding Creation Define Dataset and Dataloaders in PyTorch Neural Network definition in PyTorch The Training Loop Model Validation The idea about using Embeddings from Categorical Features was first mooted during a Kaggle contest and a paper was also published on this. In the context of NLP and word embeddings, we represent each word in an n dimesnional vector space. In a similar way, we can represent any categorical feature in an n dimesnional vector space as well. 1. Dataset Download We will utilize the UCI machine learning repo which has a dataset on credit card default for customers in Taiwan. This dataset is also av

How to use Stratified Sampling using scikit-learn and pandas

Many times, when we are working on classification problems in Data Science, we blindly perform Random Sampling and split the dataset into train and test sets. For the uninitiated, a random sampling is a technique where all observations have equal probability of getting selected. What could possibly go wrong with Random Sampling when creating models? Let's say you wanted to predict which kind of customers might default on their credit card bills. If the whole dataset had 65% male and 35% female distribution,but your test set has a distribution of 55% females, your model validation will not be robust. You must believe that in future as well your male and female distribution should be similar to 65:35 and then validate your models. So, the whole point of sampling is that you want your test set to mimic your train set in terms of similar distribution to make your models as robust as possible. Scenarios in which you want your categorical features to have a s

Keyword Arguments in Python

Keyword Arguments in Python In the previous post , we discussed about Positional Arguments in Python . In this post, let's look at Keyword Arguments in Python . We will reuse the function print_stats . def print_stats(name ="Sachin Tendulkar",age = 49,type_batsman = "Right", avg_runs=44.8): print(f"Name of cricketer is {name}.") print(f"Age of {name} is {age} years.") print(f"{name} is {type_batsman} handed bastsman.") print(f"Career ODI Average for {name} is {avg_runs} runs.") print_stats() ---> Name of cricketer is Sachin Tendulkar. Age of Sachin Tendulkar is 49 years. Sachin Tendulkar is Right handed bastsman. Career ODI Average for Sachin Tendulkar is 44.8 runs. Keyword arguments come in the form kwarg=value . Notice, how we have given values for each of our function arguments ( name ="Sachin Tendulkar" , age = 49 , type_batsman = "Right" , avg_runs=44.8 ). If t

Positional Arguments in Python

Positional Arguments in Python Positional Arguments in Python allow arguments to be passed to a function implicitly based on the order in which they appear in the function argument list. When you define a Python function, you expect some values or arguments to be passed to the function. Let's define a function to understand this better. def print_stats(name,age,type_batsman, avg_runs): print(f"Name of cricketer is {name}.") print(f"Age of {name} is {age} years.") print(f"{name} is {type_batsman} handed bastsman.") print(f"Career ODI Average for {name} is {avg_runs} runs.") In this function, name , age , type_batsman , avg_runs are called positional arguments . The function expects the order to be maintained when you pass in parameters to this function. Let's call this function to see this. print_stats("Sachin Tendulkar", 49, "right", 44.8) Name of cricketer is Sachin Te

Tutorial on Python Classes with Example

Tutorial on Python Classes with Example In the previous post , we introduced the topic of classes in Python. We will continue to develop these ideas in this post. We need to calculate the newspaper,egg and milk expenses for any person. Using this information, we will finally calculate the total expenses. Let's see this in action. Recall that our __init__ method allowed us to initialize name of the person, the kind of milk, eggs and newspaper puchased by the person. def __init__(self,name,milk_type,newspaper_type,egg_type): self.name = name self.milk_type= milk_type self.newspaper_type = newspaper_type self.egg_type = egg_type To calculate the egg expenses incurred, we define a method called calculate_egg_expenses . Recall that a function defined within a class is called a method . Code block below explains this. def calculate_egg_expenses(self,number_of_eggs_per_day): if self.egg_type =="small": self.egg_expense

How to define a Class in Python?

How to define a Class in Python? What is self and __init__ in Python? We will try to explain all this in this post. Before we explain about classes, let us make some assumptions. Imagine a scenario where you are tracking your household expenses. In partcular, milk, egg and newspaper expenses. There are 2 kinds of milk available. Let's call them blue and green . We have 2 kinds of egg available, small and large and 2 kinds of newspaper available Deccan Herald and TOI class Expenses: pass print(Expenses) class __main__.Expenses We just created a class called Expenses . The pass just tells you that we will revisit Expenses class later. Now, let's assume, I wanted to track my expenses. How do I use the Expenses class for my purpose? I need to create an instance of a class for myself. Let's do that below. Rohit_expenses = Expenses() print(Rohit_expenses) __main__.Expenses object at 0x7f74483de950> We can see that an Expenses obj

What is a set comprehension and dictionary comprehension?

What is a set comprehension and dictionary comprehension? In our earlier posts ( Part 1 and Part 2 ) , we discussed extensively on list comprehension. In this post, I would like to conclude these ideas and see how we can extend it to other iterables in Python. A set comprehension works in a similar fashion. A set data structure in Python stores unique elements. Let's create a list of numbers list_nos = [1,2,3,4,4,4,5,6,6,2,2,2,1,1,0] We create a set using set comprehension as below. set_nos = {element for element in list_nos} print(set_nos) ---> {0, 1, 2, 3, 4, 5, 6} Another example of a set comprehension can be seen below. Let's create a sentence or rather a string in Python. sentence = "I am having a great day" set_sentence = {element for element in sentence} print(set_sentence) ---> {'t', 'e', 'h', 'I', 'm', 'v', 'y', ' ', 'n', 'g', 'a'

How to flatten a list of lists using List Comprehension?

How to flatten a list of lists using List Comprehension? In the previous post , we discussed main ideas around list comprehension. We will continue to extend the ideas in this post. Let's create a list of lists first. list_of_lists = [ [ 1,2,3,4], [4,5,6], [7,8,9] ] Flattening list of lists -the Usual Way . list_flatten = [ ] for element_list in list_of_lists: for element in element_list: list_flatten.append(element) print(list_flatten) ---> [1, 2, 3, 4, 4, 5, 6, 7, 8, 9] Flatten List using List Comprehension . list_flatten_comprehension =[ element for element_list in list_of_lists for element in element_list] print(list_flatten_comprehension) --->[1, 2, 3, 4, 4, 5, 6, 7, 8, 9] Beginners in Python tend to make one mistake (At least I used to make it often). Name Error while using List Comprehension . What would happen if you write list comprehension as below? list_flatten_comprehension = [element for element i

What is a List Comprehension in Python?

Scenario 1: Let's say you have been given a list of numbers and you need to double each element in the list and store it in a new list. How would you do it? list_numbers = [1,2,3,4] list_double=[ ] for element in list_numbers: list_double.append(2*element) print(list_double) ---> [2, 4, 6, 8] Is there a better way of writing this? In a more Pythonic way? Enter List Comprehension!! list_double_with_comprehension = [2*element for element in list_numbers] print(list_double_with_comprehension) ---> [2, 4, 6, 8] Using List comprehension, a 3-line code gets converted to a 1-line code. In general, the format for a list comprehension requires at-least 2 pieces of information. The expression which needs to be computed Iterating through the iterable When we created list_double_with_comprehension, (2*element) is the expression which is computed and "for element in list_numbers" is the way we iterate through the original list ca

Iterables and Iterators- Part 2

In continuation to my previous post on Iterables , we should now get familiar with something new. Let's again create a list. lst = [1,2,3,4] dir(lst) --->['__add__', '__class__', '__contains__', '__delattr__', '__delitem__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__iadd__', '__imul__', '__init__', '__init_subclass__', '__iter__', '__le__', '__len__', '__lt__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__rmul__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', 'append', 'clear', 'copy', 'count', 'extend',

Iterables and Iterators-- Part 1

How would you generally traverse any Python data structure? Or in plain english , how do you visit each element in a Python data structure? Don't know? ... Guess what .. Use a "for loop". Now, if you can use a "for loop" to traverse through a Python data structure, that data structure is called an Iterable .. Simple!! Let's create a Python list lst = [1,2,3,4] for element in lst: print(element) ---> 1 2 3 4 Now let's create a Python set set_new = set([4,5,6,7]) for element in set_new: print(element) ---> 4 5 6 7 Let's create a Python string now strng = 'Python' for element in strng: print(element) ---> 'P' 'y' 't' 'h' 'o' 'n' If you try a similar approach with dictionaries and tuples, the effect will be the same. We call Python data structures like lists, tuples, set, string, dictionary as Itera

Negative Indices in Python

Negative Indices in Python In this post, we will go into a concept known as negative indices and negative slicing in Python. How do you utilize negative indices in Python? If you have not read my post on slicing in Python, I have given the link here. Let's create a list of numbers. list_nos = [10,20,30,40,50,60] list_nos[1:4] --->[20,30,40] list_nos[:2] ---> [10,20] Let's say, we want to access last element in list. list_nos[len(list_nos)-1] --->[60] Previous example can also be written as below list_nos[-1] ---> [60] If you want the second from last element list_nos[-2] ---> [50] If you want the third from last element list_nos[-3] ---> [40] You get the point!!! Negative index can be sometimes easier to use than positive index. If you are still confused, I just made a helper image to help understand better. How do you translate positive index to a negative index in Python ? Negative I

Slicing in Python

Slicing in Python In this post, we will understand concepts around slicing lists in Python. Let's say, I have names of my friends stored in a Python List list_friends = ["Irfan", "Jitu", "Ketan", "Ravi", "Rohan", "Zephyr"] If I ask you , give me 4 th and 5 th friend in the list, this is how it is done. list_friends[4:6] ---> ['Rohan', 'Zephyr'] If I say, give me alternate names of friends starting from Irfan. list_friends[0:5:2] ----> ['Irfan', 'Ketan', 'Rohan'] The slice operator is in the form as given below start:end:step Interpret this as "Go from start index to end index (but exclude element at end index) with a step size of step " In the previous example, we went from 0 to 5 with a step size of 2 If start index is not given, Python assumes it as 0 If end index is not give, Python assumes it as last index (equal to