Well I am actually in NYC this weekend, we booked part of Rye Castle (which is a sorta-castle overlooking a golf course overlooking the ocean) for my dear old mum's birthday.
If you want to avoid the touristy NYC, you got to get out of midtown and Broadway. Here are some of my old haunts:
As for places to go, Momofuku ramen joint at around 1st and 10th.
There is a Tapas joint right in front of it that has flamenco dancers once a week.
Now that I mention it, a stroll down St Marks is pretty cool. Would have been cooler if they didn't close down the automat.
Further down 1st avenue is a pair of Bangladeshi restaurants, notable for the crazy lighting. I swear the place is a fire hazard but it's a must visit:
For the kids, there's the Little Red Lighthouse, situated beneath George Washington Bridge. Interesting story around it. There is also the children's book about it.
A bit hokey, but I've gone on a family dinner cruise around Manhattan island. Its a good way to be acquainted with the geography
A bit more touristy is walking the High Line, which is a park converted from an abandoned rail line:
Finally, no matter how touristy it may seem, Central Park is a great family place. Your kids can go remote control sailboating in the pond