How to decide whether to move element up/down or left/right

I have a group of static bodies that represent buildings using PyBox2D, I also have dynamic circles that represent pedestrians (Top-Down visualization) I need the pedestrians to move along the building shape (the buildings are for now rectangular). That means that when they are located along horizontal side of a building I want them to move left/right and when they are located along the vertical side of a building I want them to move upwards/downwards

I came up with an algorithm that if the absolute value of difference between X-axis of building position and Y-axis position of a pedestrian is greater or equal than half of the length of building site + some tolerance it means that the pedestrian is located along vertical side and should move upwards/downwards and for another direction is inverse

# Generate Skyscrapers
for i in range(4):
    Skyscrapers.append(Building(Box_2_World,shape = (5,5), position =  Rectangles[i]))

while running    
    # Generate pedestrians
    if not Tick_Counter % FPS:

        if k <= len(Skyscrapers)-1:
            Which_Skyscraper = Skyscrapers[random.randint(0,len(Skyscrapers)-1)].position
            Random_Skyscraper = Skyscrapers[random.randint(0,len(Skyscrapers)-1)]
            Johnnie_Walkers.append(Pedestrian(Box_2_World, position = (Which_Skyscraper[0] -random.randint(-Random_Skyscraper.shape[0],Random_Skyscraper.shape[0]),Which_Skyscraper[1] -random.randint(-Random_Skyscraper.shape[1],Random_Skyscraper.shape[1]))))
            k = k+1

    # Make the pedestrian walk
    Tick_Counter = Tick_Counter + 1
    Random_Johnnie = random.randint(0,len(Johnnie_Walkers)-1)

    if abs(Skyscrapers[Random_Johnnie].position[1] -  Johnnie_Walkers[Random_Johnnie].position[0]) >= Skyscrapers[Random_Johnnie].shape[1]/2 +1:
        Johnnie_Walkers[Random_Johnnie].body.ApplyForce(b2Vec2(5,0), Johnnie_Walkers[Random_Johnnie].body.worldCenter,True)

    elif abs(Skyscrapers[Random_Johnnie].position[0] -  Johnnie_Walkers[Random_Johnnie].position[1]) >= Skyscrapers[Random_Johnnie].shape[0]/2 +1:
        Johnnie_Walkers[Random_Johnnie].body.ApplyForce(b2Vec2(0,5), Johnnie_Walkers[Random_Johnnie].body.worldCenter,True)

I have described expected results above but does not move the way it should (for example pedestrians along vertical axis move left/right)

Thank you very much