A frequent concern in applied economics is that key empirical findings may be driven by a tiny set of outliers. To perform outlier robustness checks in practical applications of instrumental variables regressions, the common practice is first to run ordinary two stage least squares and remove observations classified as outliers with residuals beyond a chosen cut-off value. Subsequently 2SLS is re-calculated based on non-outlying observations, and this procedure can also be iterated until robust results are obtained. In this paper, we analyze this simple robust algorithm asymptotically, then provide consistent estimation and valid inferential procedures on structural parameters for practical implementation given the cut-off value. Moreover, this paper provides asymptotic theory for setting the cut-off, which is chosen to control the gauge (proportion of outliers wrongly discovered). Finally, we construct a formal test of outlier robustness based on the Hausman type test statistics comparing between the ordinary and robust estimators. Asymptotics are derived under the null hypothesis that there is no contamination in the cross-sectional i.i.d. data. The established weak convergence result involves empirical processes and the fixed point arguments. Thus, this paper also proves the uniform and weak law for a new class of weighted and marked empirical processes of residuals in IVs regressions, allowing for estimation errors of structural parameters. An empirical application to Acemoglu et al. (2019) shows the utility of the proposed theory.