How to loop over set of lines, save its content to array to make calculations, then loop over next set of lines and perform same thing

When I read a .dat file, the second non-commented line [it’s an integer, I called S) indicates the number of sets present but also the number of variations per set. For example, if S is 21, there are 21 sets present of each 21 lines.

[Since I don’t have any issues about it, I’m skipping explaining the commented lines plus next 2 non-commented ones, plus a set of 2 columns]

In the .dat files, moving to the 6 columns section. For cataloguing purposes, I only want to extract the first line of the second column per set (I think even an index counter would do here, whichever is easier).
For calculations, I want to append the 4th column per set (called In) to an array. The complication is that I need to ignore (don’t append) every 21st line per set since they’re fictitious results. The array calculation would result in 1 value.

Then, I want to save the catalogued element and the array-calculated result to a new table of two columns. In the end I should have a table of 21 rows.

Also, I don’t know if it helps, but I only need to extract data from the .dat file till the S*S line (actual index line number needs to take into account initial non-commented lines plus S+2 lines), or till the 1st column is no longer equal to zero.

I’m not completely sure how to go about it since I’m new to python, but here is what I got for now. Grateful for any help :)

#Packages needed
from os import getcwd, listdir 
from astropy.table import Table
from astropy.io import ascii
from numpy import multiply
from colorama import Fore, Style #colour for inline display, https://pypi.org/project/colorama
w = Style.RESET_ALL # back to normal font

#print(getcwd())
working_directory = 'C:/Users/Rahoul/Desktop/scripts/'

for filename in listdir(working_directory):
    if filename.endswith('.dat'):

        print(Fore.MAGENTA + '~~~~~~~~ Opening .dat file ~~~~~~~~' + w)

        with open(working_directory + filename) as f:
            print("opening:", filename)

            type = str(filename[:-4])  #to be used to create name of new table 

            print(Fore.MAGENTA + '~~~~~~~~ Reading .dat file ~~~~~~~~' + w)

            lines = [l.strip() for l in f.readlines() if l[0] != '#']

            Nat = int(lines[0])
            print("Nature:", Nat)

            S = int(lines[1])
            S0  = int(lines[1])-1
            print("Real Size:", S0)


        with open(working_directory + filename) as f:    # NOTE: I don't really know why, but i need to 
                                                         #re-open the file here, if not it doesn't work below

            print(Fore.MAGENTA + '~~~~~~~~ Extracting N_G & GW ~~~~~~~~' + w)

            N_G = [] #empty array for N_G to be appended 
            GW  = [] #empty array for GW to be appended

            for i, lines in enumerate(f):
                row = lines.strip()
                columns = lines.split()

                if i > S0 and i < S + S0:

                    N_G.append(float(columns[0]))
                    GW.append(float(columns[1]))

            print("N_G =", N_G)
            print("size of N_G =", len(N_G))
            print()
            print("GW  =", GW)
            print("size of GW =", len(GW))
            print()
            weighted_G = multiply(N_G, GW)
            print("weighted G = ", weighted_G)

        with open(working_directory + filename) as f: # NOTE: I don't really know why again, but i need to 
                                                        #re-open the file here, if not it doesn't work below

            print(Fore.MAGENTA + '~~~~~~~~ Extracting i & I ~~~~~~~~' + w)

            i   = []
            I   = [] #Intensities

            table_A = Table(names=('i', 'A'))

            for i, lines in enumerate(f):
                row = lines.strip()
                columns = lines.split()

                if i >= 2*S and i < 2*S + S*S: 

                    #naming every column in case I need to do something with them in the future                 
                    m = int(columns[0])
                    illum = int(columns[1])
                    view = int(columns[2])
                    In = float(columns[3])
                    Q = float(columns[4])
                    U = float(columns[5])

                    i.append(illum[0])  #extracting 1st term of 2nd column
                    print(i)

                    #if (i+1) % 21 == 0:
                        #print('set', i+1)

                    I.append(In)   #extracting 4th column

                    print(Fore.MAGENTA + '~~~~~~~~ Calculating A ~~~~~~~~' + w)

                    #bond = 2*sum(multiply(weighted_G, I))
                    #print("bond A = ", bond)
                    #table_A.add_row(i, bond) 

                    #if (i+1) % 21 == 0:
                        #print('set', i+1)

        ##Output bond ASCII table as text file
        #ascii.write(table_A, 'bond_of'+ type +'.txt', Writer=ascii.FixedWidth, bookend=False, delimiter=None, overwrite=True)

Please find attached link to an example .dat file i’m working with: https://pastebin.com/dxRdPD9E