3

I am new to Tensorflow and machine learning. I tried to modify the basic Tensorflow example in order to mimic batched input but could not get it to converge.

If I change x_data to be within [0,1) it is able to correctly compute W though.

x_data = np.random.rand(numelements,1).astype(np.float32)

Is there something wrong with my code? Here is a copy:

import tensorflow as tf
import numpy as np

# number of training samples
numelements = 100

# define input, and labled values
# note the inptu and output are actually scalar value
#x_data = np.random.rand(numelements,1).astype(np.float32)
x_data = np.random.randint(0, 10, size=(numelements,1)).astype(np.float32)
y_data = x_data * 10

# Try to find values for W and b that compute y_data = W * x + b
x = tf.placeholder(tf.float32, [None, 1])
W = tf.Variable(tf.zeros([1]))
b = tf.Variable(tf.zeros([1]))
y = tf.mul(x, W) + b

# Minimize the mean squared errors.
loss = tf.reduce_mean(tf.square(y - y_data))
optimizer = tf.train.GradientDescentOptimizer(0.5)
train = optimizer.minimize(loss)

# Before starting, initialize the variables.  We will 'run' this first.
init = tf.global_variables_initializer()

# Launch the graph.
sess = tf.Session()
sess.run(init)

# Fit the line.
for step in range(81):
sess.run(train, feed_dict={x: x_data})
if step % 20 == 0:
    print(step, sess.run(W), sess.run(b))
sidecus
  • 732
  • 5
  • 10

1 Answers1

0

My friend helped me figure out that my Gradient Descent training rate is too high. Using tips in this post I can clearly see that the loss is getting bigger and bigger and started overflowing eventually.

I changed the learning rate to 0.005 and it started converging.

Community
  • 1
  • 1
sidecus
  • 732
  • 5
  • 10