[Computer Science] In neural networks, wouldn't a transfer function like tanh(x)+0.1x solve the problems associated with activator functions like tanh?

---

• [Computer Science] In neural networks, wouldn't a transfer function like tanh(x)+0.1x solve the problems associated with activator functions like tanh?
• Why do sites like Google and Amazon need domain specific urls (amazon.ca google.ru) instead of .com always? Is it simply a language thing?
• When was there a 'cosmic visible light' background?
• How has Hurricane Harvey managed to stay in the Southeastern Texas area so long when other Hurricanes crawling up the East Coast seem to go right through one place to the next at a quicker rate?
• Is quantum entanglement limited to two particles?
• When you mix two units of liquids of different temperature, and the apparent temperature changes, is this because the individual particles' energy changes, or is it just because the average of their energies changes?
• Does compressed air provide less buoyancy than uncompressed air?
• Why does lime juice severely burn the skin when exposed to sunlight?
• What properties of a substance give it things like taste and sound?
• How does Bremsstrahlung radiation work?
• Is there a way to convert ambient heat into electricity?
• [Cryptology] How does the Maximum Distance Separable Matrix work?
• If Matter and Antimatter behave very, very similar, can we be sure that distant galaxies aren't made of antimatter?
• If electrons surround a nucleus in a cloud, can they collide, and if they can, what happens?
• What is the current thinking on the future of uranium supply and breeder reactors?
• Why is element 18 more heavy than element 19?
• Light & Matter vs Gravity: Gravitational Slingshot?
• Are there any places on Earth where the gravitational force is more than other places?
• What happened in a molecule when an atom decay into two smaller one ?
• How has the chemical composition of air in earth's atmosphere evolved over time? If present, what were the main contributing factors?
• What is the is the source/power that keeps atoms and molecules together ?
• How does chemotherapy work on a molecular level?

[Computer Science] In neural networks, wouldn't a transfer function like tanh(x)+0.1x solve the problems associated with activator functions like tanh? Posted: 28 Aug 2017 09:06 AM PDT I am just starting to get into neural networks and surprised that much of it seems to be more art than science. ReLU are now standard because they work but I have not been shown an explanation why. Sigmoid and tanh seem to no longer be in favor due to staturation killing the gradiant back propagation. Adding a small linear term should fix that issue. You lose the nice property of being bounded between -1 and 1 but ReLU already gives that up. Tanh(x)+0.1x has a nice continuous derivative. 1-f(x)2 +0.1 and no need to define things piecewise. It still has a nice activation threshold but just doesn't saturate. Sorry if this is a dumb idea. I am just trying to understand and figure someone must have tried something like this. EDIT Thanks for the responses. It sounds like the answer is that some of my assumptions were wrong. Looks like a continuous derivative is not that important. I wanted things to be differential everywhere and thought I had read that was desirable, but looks like that is not so important. Speed of computing the transfer function seems to be far more important than I had thought. ReLU is certainly cheaper. Things like SELU and PReLU are similar which approach it from the other angle. Making ReLU continuous rather than making something like tanh() fixing the saturation/vanishing grad issues . I am still not sure why that approach is favored but probably again for speed concerns. I will probably end up having to just test tanh(x)+cx vs SELU, I will be surprised if the results are very different. If any of the ML experts out there want to collaborate/teach a physicist more about DNN send me a message. :) Thanks all. submitted by /u/f4hy [link] [comments]

Why do sites like Google and Amazon need domain specific urls (amazon.ca google.ru) instead of .com always? Is it simply a language thing? Posted: 28 Aug 2017 08:42 PM PDT submitted by /u/dorkmax [link] [comments]

When was there a 'cosmic visible light' background? Posted: 29 Aug 2017 05:18 AM PDT Since the expansion of space causes the wavelength of cosmic microwave background to lengthen, presumably it would have covered other parts of the electromagnetic spectrum in which wavelengths are shorter. We also seem to have a pretty good idea of the rate (and rate of acceleration) at which space expands. So at what point in the history of the universe could I look around me and see blue everywhere? Is there an equation for this? I'm not afraid of math. submitted by /u/BrainEnema [link] [comments]

How has Hurricane Harvey managed to stay in the Southeastern Texas area so long when other Hurricanes crawling up the East Coast seem to go right through one place to the next at a quicker rate? Posted: 28 Aug 2017 08:53 AM PDT I went through Hurricane Matthew last year, and it seemed to pass through us in about half a day. Harvey seems to be in the same general area it's been for the last 2+ days. submitted by /u/JeffTennis [link] [comments]

Is quantum entanglement limited to two particles? Posted: 28 Aug 2017 09:21 AM PDT Could two, three or millions of particles separated by space-time be "entangled"? submitted by /u/LetsSpeakAboutIt [link] [comments]

When you mix two units of liquids of different temperature, and the apparent temperature changes, is this because the individual particles' energy changes, or is it just because the average of their energies changes? Posted: 29 Aug 2017 04:49 AM PDT Hey! I understand that this is a simplification of actual thermodynamics, and I assume that the kinetic energy of each molecule/atom in the final liquid ends up being roughly the same over time (except for for example the well-known aspect of the warmest molecules floating to the top in most cases), but my real question is how quickly this transfer of energy between the individual particles happens. If you were able to measure the temperatures of all the individual particles at different times after mixing the liquids, would there be a period in which the particles actually retained the same temperature as they started with, and the apparent change would just be because the average temperature had changed? And if this is something that happens over time, how quickly does it happen? submitted by /u/Hottentott14 [link] [comments]

Does compressed air provide less buoyancy than uncompressed air? Posted: 29 Aug 2017 04:48 AM PDT If I held onto a balloon underwater it tries to float to the surface, but a scuba tank has way more air in it, so even though it's heavier shouldn't it still try to float to the surface far faster than a balloon? submitted by /u/whatthehellsteve [link] [comments]

Why does lime juice severely burn the skin when exposed to sunlight? Posted: 28 Aug 2017 08:14 PM PDT submitted by /u/iAreScurrd [link] [comments]

What properties of a substance give it things like taste and sound? Posted: 28 Aug 2017 06:06 PM PDT What part of an object's atomic structure or cellular composition give it things like taste or sound (when I tap a table it sounds different than when I tap a window)? submitted by /u/Ayaksnolkop_Ailatan [link] [comments]

How does Bremsstrahlung radiation work? Posted: 28 Aug 2017 10:58 PM PDT My understanding is that when a charged particle slows down due to an electric field it emits a photon in order to conserve energy. The example given on Wikipedia is an electron being deflected by a proton would emit a photon to conserve energy. The electron loses kinetic energy but it also gains electric potential energy so there is no energy violation. Does this imply that potential energy is not a valid energy form and can't be used for conservation of energy? Also this radiation is created in synchrotrons where the acceleration is perpendicular to the velocity. There is no change in energy on this situation so no radiation is required to conserve energy is it? Is there a gravitational equivalent of Bremsstrahlung radiation? submitted by /u/Prowsey [link] [comments]

Is there a way to convert ambient heat into electricity? Posted: 28 Aug 2017 05:49 PM PDT Say you live in a very hot area. Is there a way to use something like solar panels to absorb the heat around your house and convert it into electricity? submitted by /u/OK_Soda [link] [comments]

[Cryptology] How does the Maximum Distance Separable Matrix work? Posted: 28 Aug 2017 09:03 PM PDT To any cryptologists and other computer scientists out there, could you please tell me how the MDS matrix works? I honestly can't understand it. For context, I haven't really studied anything past confusion, diffusion, feistel networks, lai-massey, and sp-networks, and bitwise operations. submitted by /u/ThomasKingofNowhere [link] [comments]

If Matter and Antimatter behave very, very similar, can we be sure that distant galaxies aren't made of antimatter? Posted: 28 Aug 2017 01:04 PM PDT To my recollection, recent tests have shown that Anti-Hydrogen reflects light very similarly to normal hydrogen. Can we still tell them apart over astronomical distances? submitted by /u/Ten_Tacles [link] [comments]

If electrons surround a nucleus in a cloud, can they collide, and if they can, what happens? Posted: 28 Aug 2017 11:03 AM PDT submitted by /u/KPC51 [link] [comments]

What is the current thinking on the future of uranium supply and breeder reactors? Posted: 28 Aug 2017 11:12 AM PDT I've seen various estimates on the future supply of Uranium, that there might be as little as 80-200 years worth left, and that there might be so much that running out won't ever be a concern. What's the current consensus on it? Breeder reactors sound almost like a miracle technology, with the potential to extract 100 times more energy than normal reactors and massively reducing the headache of nuclear waste, but apparently 'in 2010 the International Panel on Fissile Materials said "After six decades and the expenditure of the equivalent of tens of billions of dollars, the promise of breeder reactors remains largely unfulfilled and efforts to commercialize them have been steadily cut back in most countries."'. What are the drawbacks, or why are they infeasible in reality? Even if uranium is abundant, wouldn't they be worth pursuing simply because they get more value out of whatever amount is mined? submitted by /u/daman345 [link] [comments]

Why is element 18 more heavy than element 19? Posted: 28 Aug 2017 10:04 AM PDT On the periodic table, it says element 18 weighs 39.948, and that 19 only weighs 39.0983. Why isn't 19 heavier than 18? Doesn't it have 1 more proton? submitted by /u/WashedBag [link] [comments]

Light & Matter vs Gravity: Gravitational Slingshot? Posted: 28 Aug 2017 05:08 PM PDT Under a few assumptions (some of which are inevitably going to be wrong from my unlearned point of view), it would be possible to have light photons or matter be affected by a gravitational pull such as a black hole without being pulled in completely or into orbit? If this is so, would it be possible to "slingshot maneuver" and alter its potential energy? Light is probably exempt from this due to its constant speed throughout a frame, but what about a body with mass? Could it speed up by slingshotting round a black hole? And if so, isn't it taking energy from said black hole? submitted by /u/DNRTannen [link] [comments]

Are there any places on Earth where the gravitational force is more than other places? Posted: 28 Aug 2017 02:53 PM PDT submitted by /u/Trees7861 [link] [comments]

What happened in a molecule when an atom decay into two smaller one ? Posted: 28 Aug 2017 12:34 PM PDT submitted by /u/Night_Fires [link] [comments]

How has the chemical composition of air in earth's atmosphere evolved over time? If present, what were the main contributing factors? Posted: 28 Aug 2017 01:57 PM PDT submitted by /u/Hashs1ingings1asher [link] [comments]

What is the is the source/power that keeps atoms and molecules together ? Posted: 28 Aug 2017 07:34 PM PDT What lies beneath/behind the visible material world ? submitted by /u/de-kandelaar [link] [comments]

How does chemotherapy work on a molecular level? Posted: 28 Aug 2017 10:39 AM PDT I don't know much about chemotherapy other than it is used to treat cancer, but I do have a high-school level background on chemistry and biology, so could someone explain what chemotherapy does, on a molecular level? submitted by /u/MeowrCat [link] [comments]

You are subscribed to email updates from AskScience: Got Questions? Get Answers.. To stop receiving these emails, you may unsubscribe now.	Email delivery powered by Google
Google Inc., 1600 Amphitheatre Parkway, Mountain View, CA 94043, United States