Asked my instructor and he gave me a sweep to use.
Once they have passed their knee through but you still hae their ankle trapped, lets say you have their right leg trapped, reach through their legs with your left arm and grab one of the flaps of their gi (or their belt).
You have to make sure you don't stay flat on your back and that you pull on the flap you have a hold of. The next part is to push the knee of your left leg into their butt to get them off balance a bit more and then as they pass, you use the pressure and the grip you have to follow them up to your knees.
ttt
release his grip on your collar with your both hands and go for the back.