Would you gain momentum falling towards a large asteroid if the large asteroid was moving away?

Lets say a large asteroid had boosters pushing it slowly in a certain direction, lets say on the opposite side to the direction the large asteroid was moving I jumped out of a tall building, would I speed up?

Lets say the boosters we in sync with the speed I was falling from the building from a third reference point would I be speeding up towards to the large asteroid?

This is in response to a question I asked yesterday but slightly changed so I can feel sure I understand it.