Python Dataframe Set True if last n rows are True

Question

I am trying to create a new column where in, True if last n rows are True in other column. It works fine but the problem is it is taking lot of time. This is my code: dfx = pd.DataFrame({'A':[False,False,False,False,True,True,True,True,False,True]}) n=2 ## n to cover 10 min range samples cl_id = dfx.columns.tolist().index('A') ### cl_id for index number of the column for using in .iloc l1=[False]*n+[all(dfx.iloc[x+1-n:x+1,cl_id].tolist()) for x in np.arange(n,len(dfx))] dfx['B'] = l1 print(dfx) #old_col # New_col A B 0 False False 1 False False 2 False False 3 False False 4 True False 5 True True ## Here A col last two rows True, hence True 6 True True ## Here A col last two rows True, hence True 7 True True ## Here A col last two rows True, hence True 8 False False 9 True False Can anyone suggest me a better way to achieve it? Select the correct answer from above options

Answer 1

You need to use the pandas.Series.rolling: n = 2 dfx["A"].rolling(n).sum().eq(n) Output: 0 False 1 False 2 False 3 False 4 False 5 True 6 True 7 True 8 False 9 False Name: A, dtype: bool Use the benchmark against OP (about 1000x faster): dfx = pd.DataFrame({'A':[False,False,False,False,True,True,True,True,False,True]*1000}) %timeit -n10 l1 = dfx["A"].rolling(n).sum().eq(n) # 702 µs ± 88.6 µs per loop (mean ± std. dev. of 7 runs, 10 loops each) %timeit -n10 l2 = [False]*n+[all(dfx.iloc[x+1-n:x+1,cl_id].tolist()) for x in np.arange(n,len(dfx))] # 908 ms ± 24 ms per loop (mean ± std. dev. of 7 runs, 10 loops each) l1.tolist() == l2 # True If you are a beginner and want to know more about Python the do check out the python for data science course