You get that due to defaults for pd.merge:
merge(left, right, how='inner', on=None, left_on=None, right_on=None,
left_index=False, right_index=False, sort=False, suffixes=('_x',
'_y'), copy=True, indicator=False)
on : label or list
Field names to join on. Must be found in both DataFrames. If on is
None and not merging on indexes, then it merges on the intersection of
the columns by default.
You haven't pass any key to on key, so it merges on the intersection of the columns by default. You have different indices for your df1 and df2 so if you want to keep left or right you should specify that:
In [43]: pd.merge(df1, df2)
Out[43]:
x y z
0 1 3 8
1 3 5 7
2 5 7 6
In [44]: pd.merge(df1, df2, on='y', left_index=True)
Out[44]:
x y z
c 1 3 8
d 3 5 7
e 5 7 6
In [45]: pd.merge(df1, df2, on='y', right_index=True)
Out[45]:
x y z
a 1 3 8
c 3 5 7
e 5 7 6