Here's the situation:

One of my players cast Improved Invis. on himself (and succeeded), flanked an enemy and is now engaging in melee combat with his claymore.

I rolled Initiative for both him and the NPC, and the NPC scored more hits so I assume NO SURPRISE ATTACK...

However, in actual combat, isn't the NPC unable to defend the attack as he is "unaware of the attack?"

Please help me sort this out...

First, does the NPC know the player is there, but invisible? If so, let him roll active perception (simple action, -2 distraction for combat, -6 since he can't see, +3 for active search) If he gets at least 1 hit (I assume the PC isn't actively sneaking in a combat situation), he knows generally where the character is (he can attack with the other simple, but at a -6 penalty). At least, that is what I would do. I hope it helps.

Seems reasonable to me. If it's not a surprise situation, he should have seen the caster vanish.

And by 2072 you'd think most people would be aware that magicians and/or spirits can vanish from sight and still do mean things. Heck, it's probably exaggerated on all kinds of trid shows.

I don't think -6 is appropriate. Hearing is just as valuable for perception as seeing. He could even smell him without a negative modifier. Only attacking suffers a -6 blind penalty. And most likely defending.
Defending with -6 is probably as bad as getting no defense at all due to surprise. I can see this work.

How much does hearing a sword coming from a vague direction really help though? But yea, seeing how -6 is full blind penalty I could see -5 or -4.

I suppose you would be right, after all, he's trying to find him with the other senses. However, as humans are a mostly visual species, I think there should be some penalty for having to use other senses (I wouldn't require the same for a wolf shifter or a Naga, though, as they use other dominant senses, hearing/smell and taste/smell)

After reading the spell description (SR4 not SR4a) here's how I would handle it:

Defender rolls either (Willpower (if invisibility) or Intuition (if Imp. Invis)) + Counterspelling if he rolls more hits than the spell caster he has resisted the Invisibility spell and can act normally. If the defender doesn't succeed then he attacks using the blind fire rules which means Intuition + Combat Skill - 6 (this is the standard Target Hidden rules).

(Meta)Humans don't normally use senses other than sight for targeting, our brains just aren't wired that way. That's why the "Target Hidden" rules are set up the way they are. If the defender has somehow set themselves up with say hearing or small as a targeting sense I wouldn't apply the Target Hidden rules.

Just my 2 NuYen.
-D

Exactly, however, the NPC still needs to find the invisible character, and for that, a perception test is required.

Only if the NPC doesn't resist the spell. If the spell is resisted the attacker becomes pretty obvious.

-D

Obviously. But, in the OP, he says the spell is successful, so I'm assuming that means it isn't resisted.

My bad. Way to early in the morning.

-D