Atoms are so small that any sample you can see contains an absurd number of them. If you counted a billion atoms every second, finishing one mole would take about 19 million years. NA saves you the counting: it is the number of particles in one mole, so its unit is "per mole", mol−1.
Treat it like any unit conversion and the direction takes care of itself. From moles to particles you multiply: 0.250 mol×6.022×1023 mol−1=1.51×1023 particles, and the mol units cancel. From particles to moles you divide: 3.0×1022÷(6.022×1023)=0.050 mol. A sample you can hold has a huge particle count and a modest number of moles, so a result like 4×10−24 molecules means the conversion was flipped.
NA counts the entity you name, not automatically atoms. One mole of O2 is 6.022×1023 molecules but 1.204×1024 atoms. One mole of CaCl2 is 6.022×1023 formula units, which break apart in water into three times as many ions. Molar mass is a different conversion: it links moles to grams, while NA links moles to particles. A problem that starts in grams and ends in particles needs both.
Think of it like this. Avogadro's number is an exchange rate, like 100 cents to the dollar but vastly larger: one mole trades for 6.022×1023 particles. Going from the big unit to the small one, you multiply; coming back, you divide. Unlike money, you must say which particle you are trading in, since one mole of molecules can contain several moles of atoms.
Core idea. Avogadro's number, 6.022×1023 mol−1, is the conversion factor between an amount in moles and an actual count of particles. It works for any entity, so the particle you specify decides what the count means.