The kind of after-effects, indicative of crossmodal recalibration that are observed after exposure to spatially incongruent inputs from different sensory modalities have not been demonstrated so far for identity incongruence. We show that exposure to incongruent audiovisual speech (producing the well-known McGurk effect) can recalibrate auditory speech identification. Exposure to an ambiguous sound, intermediate between /aba/ and /ada/, dubbed onto vision of a face articulating either /aba/ (or /ada/), increased the proportion of /aba/ (or /ada/) responses during subsequent sound identification trials. In contrast, fewer /aba/ (or /ada/) responses occurred when a congruent non-ambiguous sound was dubbed onto the face articulating /aba/ (or /ada/), revealing selective speech adaptation. When submitted to separate forced-choice identification trials, the bimodal stimulus pairs producing these contrasting effects were identically categorized, which makes a role of post-perceptual factors in the generation of these effects unlikely.