Oh poor you. That must be very frustrating for you.
My DD wasn't interested in a sippy cup but did learn how to use a straw and will now take a sippy with water if necessary.
As for the finger foods - I guess you've tried everything!! But my one thought was have you tried giving them to him not at meal times? Rather as a little snack. So he's less hungry and there's less 'pressure' to eat.
You say you've got an older LO - my two love carpet picnics (or garden ones when it's not raining!); strawberries, blueberries, bits of cheese, ripe mango, oat and banana bars, corn snacks, are all favourites here and (particularly if we're in the garden) they both wander around while they're eating.